2021-02-20T07:55:34.864 INFO:root:teuthology version: 1.1.0-9a6b849c 2021-02-20T07:55:34.867 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_trociny@teuthology --verbose --description rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} --archive /home/teuthworker/archive/trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi/5897918 --name trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi -- /home/teuthworker/archive/trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi/5897918/orig.config.yaml 2021-02-20T07:55:34.932 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-02-20T07:55:35.008 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi/5897918 branch: wip-mgolub-testing description: rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} email: to.my.trociny@gmail.com first_in_suite: false job_id: '5897918' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: run two ceph clusters and install rbd-mirror - desc: configure the permissions for client.mirror - desc: 2 ceph clusters with 1 mon and 3 osds each - desc: run four rbd-mirror daemons per cluster - desc: run the rbd_mirror_stress.sh workunit to test the rbd-mirror daemon name: trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 3 size: 30 os_type: ubuntu os_version: '18.04' overrides: admin_socket: branch: wip-mgolub-testing ceph: conf: client: debug rbd: 20 debug rbd_mirror: 15 log to stderr: false rbd default features: 125 rbd mirror image policy type: none client.mirror.0: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.1: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.2: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.3: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid global: mon client directed command retry: 5 ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd memory target: 1610612736 osd objectstore: bluestore fs: xfs log-ignorelist: - \(OSD_SLOW_PING_TIME log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: d318d42d502ea13c78f2af2499efd19dcc7e174b ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: d318d42d502ea13c78f2af2499efd19dcc7e174b thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mgolub-testing sha1: d318d42d502ea13c78f2af2499efd19dcc7e174b owner: scheduled_trociny@teuthology priority: 50 repo: https://github.com/ceph/ceph-ci.git roles: - - cluster1.mon.a - cluster1.mgr.x - cluster2.mgr.x - cluster1.osd.0 - cluster1.osd.1 - cluster1.osd.2 - cluster1.client.0 - cluster2.client.0 - - cluster2.mon.a - cluster2.osd.0 - cluster2.osd.1 - cluster2.osd.2 - cluster1.client.mirror - cluster1.client.mirror.0 - cluster1.client.mirror.1 - cluster1.client.mirror.2 - cluster1.client.mirror.3 - cluster1.client.mirror.4 - cluster1.client.mirror.5 - cluster1.client.mirror.6 - cluster2.client.mirror - cluster2.client.mirror.0 - cluster2.client.mirror.1 - cluster2.client.mirror.2 - cluster2.client.mirror.3 - cluster2.client.mirror.4 - cluster2.client.mirror.5 - cluster2.client.mirror.6 sha1: d318d42d502ea13c78f2af2499efd19dcc7e174b sleep_before_teardown: 0 suite: rbd suite_branch: wip-mgolub-testing suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_d318d42d502ea13c78f2af2499efd19dcc7e174b/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: d318d42d502ea13c78f2af2499efd19dcc7e174b targets: smithi003.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyx++l8vCCc5LmHtTy8pQxD4u432ooKGE/AHMpohypjtB5vWw9eqB1m8w0iOU+wGGonfdzP49qcnKxw7NUtQh2vSkRA2kXw+sz6IzEsm7eqVV2UypIjsbl09hZB4m94uYSl5urqZ3IbVMVYUDtH+NAU8I+ghkhkTRSnb8Lc4bQmOiKDRyNXAa0mdofbiTcj32PSs8iWuDN2imSWHlUeBFjOvDKD1IGBic/qXPgTjWhDVTGyyT1GNM1Sv5pTbmpxLyxjm1qTMHVpU1nFgQTlUW4AJdmqF5ASBFlNJJjCxGguxlXAqqCbbz5cgzYnB1bwf7IzOtHBonnYp7ydVH0JDjB smithi090.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJHRcfKBV/OkZP8o1/N7APaJg3UhsSRUYymD9kh3n2b7ZdoWr7TS7S8ZaUjR/zdYkdiX/JxbVf/WciS0KJ02FYSNpPTMHOgeqyw1Ih37MiOXKzwsg1PxdZmiqyNGkAlqr5pY61dA6xWSPgmvbvrmZlHkX97KC5EBl7gsvJokmA3YidM5VbGerWOLI/bRmbpzi8wJh9Ovm+rbln7Es0Jh7QBARXsFU9QNocjPWXhbHv7049UrfKUbAfiyBRpyFx6cePrOWl5P7lqO1h90JO8LxUa5c8ByjRGGE7PcMFIcQq0TyGAgSPnLdU4b6UM3z6FNPKL4pxeR1CuyiHjS5JF7XL tasks: - install: extra_packages: - rbd-mirror - ceph: cluster: cluster1 - ceph: cluster: cluster2 - exec: cluster1.client.mirror.0: - sudo ceph --cluster cluster1 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd' - rbd-mirror: client: cluster1.client.mirror.0 thrash: true - rbd-mirror: client: cluster1.client.mirror.1 thrash: true - rbd-mirror: client: cluster1.client.mirror.2 thrash: true - rbd-mirror: client: cluster1.client.mirror.3 thrash: true - rbd-mirror: client: cluster2.client.mirror.0 thrash: true - rbd-mirror: client: cluster2.client.mirror.1 thrash: true - rbd-mirror: client: cluster2.client.mirror.2 thrash: true - rbd-mirror: client: cluster2.client.mirror.3 thrash: true - rbd-mirror-thrash: cluster: cluster1 - rbd-mirror-thrash: cluster: cluster2 - workunit: clients: cluster1.client.mirror: - rbd/rbd_mirror_stress.sh env: CEPH_ARGS: '' MIRROR_IMAGE_MODE: snapshot MIRROR_POOL_MODE: image RBD_IMAGE_FEATURES: layering,exclusive-lock,object-map,fast-diff RBD_MIRROR_INSTANCES: '4' RBD_MIRROR_USE_EXISTING_CLUSTER: '1' RBD_MIRROR_USE_RBD_MIRROR: '1' timeout: 6h teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.29353 2021-02-20T07:55:35.009 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_d318d42d502ea13c78f2af2499efd19dcc7e174b/qa; will attempt to use it 2021-02-20T07:55:35.010 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_d318d42d502ea13c78f2af2499efd19dcc7e174b/qa/tasks 2021-02-20T07:55:35.011 INFO:teuthology.run_tasks:Running task internal.check_packages... 2021-02-20T07:55:35.013 INFO:teuthology.task.internal:Checking packages... 2021-02-20T07:55:35.039 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'basic' and ceph hash 'd318d42d502ea13c78f2af2499efd19dcc7e174b' 2021-02-20T07:55:35.040 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-02-20T07:55:35.041 INFO:teuthology.packaging:ref: None 2021-02-20T07:55:35.041 INFO:teuthology.packaging:tag: None 2021-02-20T07:55:35.041 INFO:teuthology.packaging:branch: wip-mgolub-testing 2021-02-20T07:55:35.042 INFO:teuthology.packaging:sha1: d318d42d502ea13c78f2af2499efd19dcc7e174b 2021-02-20T07:55:35.042 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=wip-mgolub-testing 2021-02-20T07:55:35.220 INFO:teuthology.task.internal:Found packages for ceph version 17.0.0-929-gd318d42d-1bionic 2021-02-20T07:55:35.221 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2021-02-20T07:55:35.230 INFO:teuthology.task.internal:no buildpackages task found 2021-02-20T07:55:35.231 INFO:teuthology.run_tasks:Running task internal.save_config... 2021-02-20T07:55:35.244 INFO:teuthology.task.internal:Saving configuration 2021-02-20T07:55:35.264 INFO:teuthology.run_tasks:Running task internal.check_lock... 2021-02-20T07:55:35.277 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-20T07:55:35.297 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-02-20 07:49:27.635704', 'description': '/home/teuthworker/archive/trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi/5897918', 'locked_by': 'scheduled_trociny@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILiBqjxdSUCkV0OSU0iBXSKm6ssgs95RzvT5JNJ/NSWB', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi003.front.sepia.ceph.com'} 2021-02-20T07:55:35.314 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-02-20 07:49:27.635683', 'description': '/home/teuthworker/archive/trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi/5897918', 'locked_by': 'scheduled_trociny@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHVmJDz8HQLFPhhDyCZF6w51lscpc35fIsEWPTiYrWcr', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi090.front.sepia.ceph.com'} 2021-02-20T07:55:35.315 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2021-02-20T07:55:35.326 INFO:teuthology.task.internal:roles: ubuntu@smithi003.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'] 2021-02-20T07:55:35.326 INFO:teuthology.task.internal:roles: ubuntu@smithi090.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6'] 2021-02-20T07:55:35.326 INFO:teuthology.run_tasks:Running task console_log... 2021-02-20T07:55:35.398 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7b32266268>, signals=[15]) 2021-02-20T07:55:35.400 INFO:teuthology.run_tasks:Running task internal.connect... 2021-02-20T07:55:35.411 INFO:teuthology.task.internal:Opening connections... 2021-02-20T07:55:35.411 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi003.front.sepia.ceph.com 2021-02-20T07:55:35.413 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T07:55:35.490 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi090.front.sepia.ceph.com 2021-02-20T07:55:35.491 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T07:55:35.567 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2021-02-20T07:55:35.575 DEBUG:teuthology.orchestra.run.smithi003:> uname -m 2021-02-20T07:55:35.651 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64 2021-02-20T07:55:35.651 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2021-02-20T07:55:35.699 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="Ubuntu" 2021-02-20T07:55:35.700 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-02-20T07:55:35.700 INFO:teuthology.orchestra.run.smithi003.stdout:ID=ubuntu 2021-02-20T07:55:35.700 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE=debian 2021-02-20T07:55:35.700 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-02-20T07:55:35.701 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="18.04" 2021-02-20T07:55:35.701 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://www.ubuntu.com/" 2021-02-20T07:55:35.701 INFO:teuthology.orchestra.run.smithi003.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-02-20T07:55:35.701 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-02-20T07:55:35.702 INFO:teuthology.orchestra.run.smithi003.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-02-20T07:55:35.702 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_CODENAME=bionic 2021-02-20T07:55:35.702 INFO:teuthology.orchestra.run.smithi003.stdout:UBUNTU_CODENAME=bionic 2021-02-20T07:55:35.703 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2021-02-20T07:55:35.719 DEBUG:teuthology.orchestra.run.smithi090:> uname -m 2021-02-20T07:55:35.767 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64 2021-02-20T07:55:35.767 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2021-02-20T07:55:35.814 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="Ubuntu" 2021-02-20T07:55:35.814 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-02-20T07:55:35.815 INFO:teuthology.orchestra.run.smithi090.stdout:ID=ubuntu 2021-02-20T07:55:35.815 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE=debian 2021-02-20T07:55:35.815 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-02-20T07:55:35.815 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="18.04" 2021-02-20T07:55:35.815 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://www.ubuntu.com/" 2021-02-20T07:55:35.816 INFO:teuthology.orchestra.run.smithi090.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-02-20T07:55:35.816 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-02-20T07:55:35.816 INFO:teuthology.orchestra.run.smithi090.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-02-20T07:55:35.816 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_CODENAME=bionic 2021-02-20T07:55:35.816 INFO:teuthology.orchestra.run.smithi090.stdout:UBUNTU_CODENAME=bionic 2021-02-20T07:55:35.817 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2021-02-20T07:55:35.836 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2021-02-20T07:55:35.849 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2021-02-20T07:55:35.864 INFO:teuthology.task.internal:Checking for old test directory... 2021-02-20T07:55:35.865 DEBUG:teuthology.orchestra.run.smithi003:> test '!' -e /home/ubuntu/cephtest 2021-02-20T07:55:35.867 DEBUG:teuthology.orchestra.run.smithi090:> test '!' -e /home/ubuntu/cephtest 2021-02-20T07:55:35.874 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2021-02-20T07:55:35.888 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2021-02-20T07:55:35.888 DEBUG:teuthology.orchestra.run.smithi003:> test -z $(ls -A /var/lib/ceph) 2021-02-20T07:55:35.917 DEBUG:teuthology.orchestra.run.smithi090:> test -z $(ls -A /var/lib/ceph) 2021-02-20T07:55:35.935 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2021-02-20T07:55:35.989 INFO:teuthology.run_tasks:Running task kernel... 2021-02-20T07:55:36.017 DEBUG:teuthology.task.kernel:normalized config {'cluster1.mon.a': {'kdb': True, 'sha1': 'distro'}, 'cluster1.mgr.x': {'kdb': True, 'sha1': 'distro'}, 'cluster2.mgr.x': {'kdb': True, 'sha1': 'distro'}, 'cluster1.osd.0': {'kdb': True, 'sha1': 'distro'}, 'cluster1.osd.1': {'kdb': True, 'sha1': 'distro'}, 'cluster1.osd.2': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.0': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.0': {'kdb': True, 'sha1': 'distro'}, 'cluster2.mon.a': {'kdb': True, 'sha1': 'distro'}, 'cluster2.osd.0': {'kdb': True, 'sha1': 'distro'}, 'cluster2.osd.1': {'kdb': True, 'sha1': 'distro'}, 'cluster2.osd.2': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.0': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.1': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.2': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.3': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.4': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.5': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.6': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.0': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.1': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.2': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.3': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.4': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.5': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.6': {'kdb': True, 'sha1': 'distro'}} 2021-02-20T07:55:36.018 INFO:teuthology.task.kernel:config {'cluster1.mon.a': {'kdb': True, 'sha1': 'distro'}, 'cluster2.mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2021-02-20T07:55:36.018 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2021-02-20T07:55:36.024 INFO:teuthology.orchestra.run.smithi003.stdout:4.15.0-134-generic 2021-02-20T07:55:36.025 INFO:teuthology.task.kernel:Running kernel on smithi003: 4.15.0-134-generic 2021-02-20T07:55:36.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo apt-get clean 2021-02-20T07:55:36.142 DEBUG:teuthology.orchestra.run.smithi003:> sudo apt-get update 2021-02-20T07:55:36.395 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-02-20T07:55:36.397 INFO:teuthology.orchestra.run.smithi003.stdout:Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2021-02-20T07:55:36.456 INFO:teuthology.orchestra.run.smithi003.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-02-20T07:55:36.706 INFO:teuthology.orchestra.run.smithi003.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-02-20T07:55:36.804 INFO:teuthology.orchestra.run.smithi003.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,544 kB] 2021-02-20T07:55:36.932 INFO:teuthology.orchestra.run.smithi003.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,885 kB] 2021-02-20T07:55:37.250 INFO:teuthology.orchestra.run.smithi003.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [916 kB] 2021-02-20T07:55:37.269 INFO:teuthology.orchestra.run.smithi003.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [298 kB] 2021-02-20T07:55:37.275 INFO:teuthology.orchestra.run.smithi003.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [226 kB] 2021-02-20T07:55:37.281 INFO:teuthology.orchestra.run.smithi003.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [29.9 kB] 2021-02-20T07:55:37.284 INFO:teuthology.orchestra.run.smithi003.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [975 kB] 2021-02-20T07:55:37.335 INFO:teuthology.orchestra.run.smithi003.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,220 kB] 2021-02-20T07:55:37.356 INFO:teuthology.orchestra.run.smithi003.stdout:Get:13 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,109 kB] 2021-02-20T07:55:37.365 INFO:teuthology.orchestra.run.smithi003.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [390 kB] 2021-02-20T07:55:37.372 INFO:teuthology.orchestra.run.smithi003.stdout:Get:15 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [248 kB] 2021-02-20T07:55:37.375 INFO:teuthology.orchestra.run.smithi003.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [247 kB] 2021-02-20T07:55:37.382 INFO:teuthology.orchestra.run.smithi003.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [33.4 kB] 2021-02-20T07:55:37.383 INFO:teuthology.orchestra.run.smithi003.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,718 kB] 2021-02-20T07:55:37.455 INFO:teuthology.orchestra.run.smithi003.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,560 kB] 2021-02-20T07:55:37.492 INFO:teuthology.orchestra.run.smithi003.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [363 kB] 2021-02-20T07:55:37.500 INFO:teuthology.orchestra.run.smithi003.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [14.3 kB] 2021-02-20T07:55:37.500 INFO:teuthology.orchestra.run.smithi003.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [31.8 kB] 2021-02-20T07:55:37.501 INFO:teuthology.orchestra.run.smithi003.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [7,300 B] 2021-02-20T07:55:39.038 INFO:teuthology.orchestra.run.smithi003.stdout:Fetched 13.1 MB in 3s (4,926 kB/s) 2021-02-20T07:55:40.502 INFO:teuthology.orchestra.run.smithi003.stdout:Reading package lists... 2021-02-20T07:55:40.551 DEBUG:teuthology.orchestra.run.smithi003:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-02-20T07:55:40.633 INFO:teuthology.orchestra.run.smithi003.stdout:Reading package lists... 2021-02-20T07:55:40.883 INFO:teuthology.orchestra.run.smithi003.stdout:Building dependency tree... 2021-02-20T07:55:40.884 INFO:teuthology.orchestra.run.smithi003.stdout:Reading state information... 2021-02-20T07:55:40.973 INFO:teuthology.orchestra.run.smithi003.stderr:E: Unable to locate package linux-image-current-generic 2021-02-20T07:55:40.975 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-02-20T07:55:40.976 DEBUG:teuthology.orchestra.run.smithi003:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-02-20T07:55:41.056 INFO:teuthology.orchestra.run.smithi003.stdout:Reading package lists... 2021-02-20T07:55:41.314 INFO:teuthology.orchestra.run.smithi003.stdout:Building dependency tree... 2021-02-20T07:55:41.315 INFO:teuthology.orchestra.run.smithi003.stdout:Reading state information... 2021-02-20T07:55:41.505 INFO:teuthology.orchestra.run.smithi003.stdout:The following additional packages will be installed: 2021-02-20T07:55:41.505 INFO:teuthology.orchestra.run.smithi003.stdout: linux-generic linux-headers-4.15.0-135 linux-headers-4.15.0-135-generic 2021-02-20T07:55:41.506 INFO:teuthology.orchestra.run.smithi003.stdout: linux-headers-generic linux-image-4.15.0-135-generic 2021-02-20T07:55:41.506 INFO:teuthology.orchestra.run.smithi003.stdout: linux-modules-4.15.0-135-generic linux-modules-extra-4.15.0-135-generic 2021-02-20T07:55:41.507 INFO:teuthology.orchestra.run.smithi003.stdout:Suggested packages: 2021-02-20T07:55:41.508 INFO:teuthology.orchestra.run.smithi003.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2021-02-20T07:55:41.508 INFO:teuthology.orchestra.run.smithi003.stdout:Recommended packages: 2021-02-20T07:55:41.508 INFO:teuthology.orchestra.run.smithi003.stdout: thermald 2021-02-20T07:55:41.541 INFO:teuthology.orchestra.run.smithi003.stdout:The following NEW packages will be installed: 2021-02-20T07:55:41.542 INFO:teuthology.orchestra.run.smithi003.stdout: linux-headers-4.15.0-135 linux-headers-4.15.0-135-generic 2021-02-20T07:55:41.543 INFO:teuthology.orchestra.run.smithi003.stdout: linux-image-4.15.0-135-generic linux-modules-4.15.0-135-generic 2021-02-20T07:55:41.543 INFO:teuthology.orchestra.run.smithi003.stdout: linux-modules-extra-4.15.0-135-generic 2021-02-20T07:55:41.544 INFO:teuthology.orchestra.run.smithi003.stdout:The following packages will be upgraded: 2021-02-20T07:55:41.544 INFO:teuthology.orchestra.run.smithi003.stdout: linux-generic linux-headers-generic linux-image-generic 2021-02-20T07:55:41.721 INFO:teuthology.orchestra.run.smithi003.stdout:3 upgraded, 5 newly installed, 0 to remove and 38 not upgraded. 2021-02-20T07:55:41.721 INFO:teuthology.orchestra.run.smithi003.stdout:Need to get 66.0 MB of archives. 2021-02-20T07:55:41.721 INFO:teuthology.orchestra.run.smithi003.stdout:After this operation, 334 MB of additional disk space will be used. 2021-02-20T07:55:41.721 INFO:teuthology.orchestra.run.smithi003.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-135-generic amd64 4.15.0-135.139 [13.1 MB] 2021-02-20T07:55:42.701 INFO:teuthology.orchestra.run.smithi003.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-135-generic amd64 4.15.0-135.139 [8,019 kB] 2021-02-20T07:55:42.938 INFO:teuthology.orchestra.run.smithi003.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-135-generic amd64 4.15.0-135.139 [32.8 MB] 2021-02-20T07:55:44.217 INFO:teuthology.orchestra.run.smithi003.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.135.122 [1,868 B] 2021-02-20T07:55:44.217 INFO:teuthology.orchestra.run.smithi003.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.135.122 [2,592 B] 2021-02-20T07:55:44.218 INFO:teuthology.orchestra.run.smithi003.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-135 all 4.15.0-135.139 [10.9 MB] 2021-02-20T07:55:44.562 INFO:teuthology.orchestra.run.smithi003.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-135-generic amd64 4.15.0-135.139 [1,097 kB] 2021-02-20T07:55:44.579 INFO:teuthology.orchestra.run.smithi003.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.135.122 [2,488 B] 2021-02-20T07:55:45.085 INFO:teuthology.orchestra.run.smithi003.stdout:Fetched 66.0 MB in 3s (21.8 MB/s) 2021-02-20T07:55:45.254 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package linux-modules-4.15.0-135-generic. 2021-02-20T07:55:45.291 INFO:teuthology.orchestra.run.smithi003.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102079 files and directories currently installed.) 2021-02-20T07:55:45.294 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../0-linux-modules-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T07:55:45.342 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking linux-modules-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:55:47.001 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package linux-image-4.15.0-135-generic. 2021-02-20T07:55:47.015 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../1-linux-image-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T07:55:47.107 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking linux-image-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:55:47.613 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-135-generic. 2021-02-20T07:55:47.624 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T07:55:47.663 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking linux-modules-extra-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:55:51.473 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../3-linux-generic_4.15.0.135.122_amd64.deb ... 2021-02-20T07:55:51.599 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking linux-generic (4.15.0.135.122) over (4.15.0.134.121) ... 2021-02-20T07:55:51.916 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.135.122_amd64.deb ... 2021-02-20T07:55:52.036 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking linux-image-generic (4.15.0.135.122) over (4.15.0.134.121) ... 2021-02-20T07:55:52.347 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package linux-headers-4.15.0-135. 2021-02-20T07:55:52.359 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../5-linux-headers-4.15.0-135_4.15.0-135.139_all.deb ... 2021-02-20T07:55:52.397 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking linux-headers-4.15.0-135 (4.15.0-135.139) ... 2021-02-20T07:55:57.165 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package linux-headers-4.15.0-135-generic. 2021-02-20T07:55:57.185 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../6-linux-headers-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T07:55:57.223 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking linux-headers-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:55:59.780 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.135.122_amd64.deb ... 2021-02-20T07:55:59.902 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking linux-headers-generic (4.15.0.135.122) over (4.15.0.134.121) ... 2021-02-20T07:56:00.223 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up linux-modules-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:56:00.774 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up linux-headers-4.15.0-135 (4.15.0-135.139) ... 2021-02-20T07:56:00.900 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up linux-image-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:56:01.346 INFO:teuthology.orchestra.run.smithi003.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-135-generic 2021-02-20T07:56:01.347 INFO:teuthology.orchestra.run.smithi003.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-135-generic 2021-02-20T07:56:01.477 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up linux-modules-extra-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:56:01.953 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up linux-headers-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:56:02.054 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up linux-image-generic (4.15.0.135.122) ... 2021-02-20T07:56:02.180 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up linux-headers-generic (4.15.0.135.122) ... 2021-02-20T07:56:02.306 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up linux-generic (4.15.0.135.122) ... 2021-02-20T07:56:02.483 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for linux-image-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:56:02.678 INFO:teuthology.orchestra.run.smithi003.stdout:/etc/kernel/postinst.d/initramfs-tools: 2021-02-20T07:56:02.678 INFO:teuthology.orchestra.run.smithi003.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-135-generic 2021-02-20T07:56:17.739 INFO:teuthology.orchestra.run.smithi003.stdout:/etc/kernel/postinst.d/zz-update-grub: 2021-02-20T07:56:17.740 INFO:teuthology.orchestra.run.smithi003.stdout:Sourcing file `/etc/default/grub' 2021-02-20T07:56:17.782 INFO:teuthology.orchestra.run.smithi003.stdout:Generating grub configuration file ... 2021-02-20T07:56:18.131 INFO:teuthology.orchestra.run.smithi003.stdout:Found linux image: /boot/vmlinuz-4.15.0-135-generic 2021-02-20T07:56:18.157 INFO:teuthology.orchestra.run.smithi003.stdout:Found initrd image: /boot/initrd.img-4.15.0-135-generic 2021-02-20T07:56:18.476 INFO:teuthology.orchestra.run.smithi003.stdout:Found linux image: /boot/vmlinuz-4.15.0-134-generic 2021-02-20T07:56:18.517 INFO:teuthology.orchestra.run.smithi003.stdout:Found initrd image: /boot/initrd.img-4.15.0-134-generic 2021-02-20T07:56:19.377 INFO:teuthology.orchestra.run.smithi003.stdout:done 2021-02-20T07:56:19.805 DEBUG:teuthology.orchestra.run.smithi003:> dpkg -s linux-image-generic 2021-02-20T07:56:19.828 INFO:teuthology.orchestra.run.smithi003.stdout:Package: linux-image-generic 2021-02-20T07:56:19.829 INFO:teuthology.orchestra.run.smithi003.stdout:Status: install ok installed 2021-02-20T07:56:19.829 INFO:teuthology.orchestra.run.smithi003.stdout:Priority: optional 2021-02-20T07:56:19.829 INFO:teuthology.orchestra.run.smithi003.stdout:Section: kernel 2021-02-20T07:56:19.829 INFO:teuthology.orchestra.run.smithi003.stdout:Installed-Size: 17 2021-02-20T07:56:19.829 INFO:teuthology.orchestra.run.smithi003.stdout:Maintainer: Ubuntu Kernel Team 2021-02-20T07:56:19.830 INFO:teuthology.orchestra.run.smithi003.stdout:Architecture: amd64 2021-02-20T07:56:19.830 INFO:teuthology.orchestra.run.smithi003.stdout:Source: linux-meta 2021-02-20T07:56:19.830 INFO:teuthology.orchestra.run.smithi003.stdout:Version: 4.15.0.135.122 2021-02-20T07:56:19.830 INFO:teuthology.orchestra.run.smithi003.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-02-20T07:56:19.831 INFO:teuthology.orchestra.run.smithi003.stdout:Depends: linux-image-4.15.0-135-generic, linux-modules-extra-4.15.0-135-generic, linux-firmware, intel-microcode, amd64-microcode 2021-02-20T07:56:19.831 INFO:teuthology.orchestra.run.smithi003.stdout:Recommends: thermald 2021-02-20T07:56:19.831 INFO:teuthology.orchestra.run.smithi003.stdout:Description: Generic Linux kernel image 2021-02-20T07:56:19.831 INFO:teuthology.orchestra.run.smithi003.stdout: This package will always depend on the latest generic kernel image 2021-02-20T07:56:19.831 INFO:teuthology.orchestra.run.smithi003.stdout: available. 2021-02-20T07:56:19.832 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-134-generic Expected: 4.15.0-135-generic 2021-02-20T07:56:19.832 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2021-02-20T07:56:19.839 INFO:teuthology.orchestra.run.smithi090.stdout:4.15.0-134-generic 2021-02-20T07:56:19.839 INFO:teuthology.task.kernel:Running kernel on smithi090: 4.15.0-134-generic 2021-02-20T07:56:19.839 DEBUG:teuthology.orchestra.run.smithi090:> sudo apt-get clean 2021-02-20T07:56:19.943 DEBUG:teuthology.orchestra.run.smithi090:> sudo apt-get update 2021-02-20T07:56:20.085 INFO:teuthology.orchestra.run.smithi090.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2021-02-20T07:56:20.187 INFO:teuthology.orchestra.run.smithi090.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-02-20T07:56:20.265 INFO:teuthology.orchestra.run.smithi090.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-02-20T07:56:20.358 INFO:teuthology.orchestra.run.smithi090.stdout:Get:4 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,544 kB] 2021-02-20T07:56:20.503 INFO:teuthology.orchestra.run.smithi090.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [916 kB] 2021-02-20T07:56:20.520 INFO:teuthology.orchestra.run.smithi090.stdout:Get:6 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [298 kB] 2021-02-20T07:56:20.525 INFO:teuthology.orchestra.run.smithi090.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-02-20T07:56:20.525 INFO:teuthology.orchestra.run.smithi090.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [226 kB] 2021-02-20T07:56:20.529 INFO:teuthology.orchestra.run.smithi090.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [29.9 kB] 2021-02-20T07:56:20.530 INFO:teuthology.orchestra.run.smithi090.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,109 kB] 2021-02-20T07:56:20.546 INFO:teuthology.orchestra.run.smithi090.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [975 kB] 2021-02-20T07:56:20.556 INFO:teuthology.orchestra.run.smithi090.stdout:Get:12 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [248 kB] 2021-02-20T07:56:20.675 INFO:teuthology.orchestra.run.smithi090.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,220 kB] 2021-02-20T07:56:21.007 INFO:teuthology.orchestra.run.smithi090.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,885 kB] 2021-02-20T07:56:21.106 INFO:teuthology.orchestra.run.smithi090.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [390 kB] 2021-02-20T07:56:21.123 INFO:teuthology.orchestra.run.smithi090.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [247 kB] 2021-02-20T07:56:21.133 INFO:teuthology.orchestra.run.smithi090.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [33.4 kB] 2021-02-20T07:56:21.136 INFO:teuthology.orchestra.run.smithi090.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,560 kB] 2021-02-20T07:56:21.172 INFO:teuthology.orchestra.run.smithi090.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,718 kB] 2021-02-20T07:56:21.227 INFO:teuthology.orchestra.run.smithi090.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [363 kB] 2021-02-20T07:56:21.233 INFO:teuthology.orchestra.run.smithi090.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [31.8 kB] 2021-02-20T07:56:21.234 INFO:teuthology.orchestra.run.smithi090.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [14.3 kB] 2021-02-20T07:56:21.234 INFO:teuthology.orchestra.run.smithi090.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [7,300 B] 2021-02-20T07:56:22.293 INFO:teuthology.orchestra.run.smithi090.stdout:Fetched 13.1 MB in 2s (6,180 kB/s) 2021-02-20T07:56:23.793 INFO:teuthology.orchestra.run.smithi090.stdout:Reading package lists... 2021-02-20T07:56:23.841 DEBUG:teuthology.orchestra.run.smithi090:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-02-20T07:56:23.923 INFO:teuthology.orchestra.run.smithi090.stdout:Reading package lists... 2021-02-20T07:56:24.181 INFO:teuthology.orchestra.run.smithi090.stdout:Building dependency tree... 2021-02-20T07:56:24.182 INFO:teuthology.orchestra.run.smithi090.stdout:Reading state information... 2021-02-20T07:56:24.262 INFO:teuthology.orchestra.run.smithi090.stderr:E: Unable to locate package linux-image-current-generic 2021-02-20T07:56:24.263 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-02-20T07:56:24.264 DEBUG:teuthology.orchestra.run.smithi090:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-02-20T07:56:24.341 INFO:teuthology.orchestra.run.smithi090.stdout:Reading package lists... 2021-02-20T07:56:24.578 INFO:teuthology.orchestra.run.smithi090.stdout:Building dependency tree... 2021-02-20T07:56:24.578 INFO:teuthology.orchestra.run.smithi090.stdout:Reading state information... 2021-02-20T07:56:24.753 INFO:teuthology.orchestra.run.smithi090.stdout:The following additional packages will be installed: 2021-02-20T07:56:24.754 INFO:teuthology.orchestra.run.smithi090.stdout: linux-generic linux-headers-4.15.0-135 linux-headers-4.15.0-135-generic 2021-02-20T07:56:24.754 INFO:teuthology.orchestra.run.smithi090.stdout: linux-headers-generic linux-image-4.15.0-135-generic 2021-02-20T07:56:24.754 INFO:teuthology.orchestra.run.smithi090.stdout: linux-modules-4.15.0-135-generic linux-modules-extra-4.15.0-135-generic 2021-02-20T07:56:24.756 INFO:teuthology.orchestra.run.smithi090.stdout:Suggested packages: 2021-02-20T07:56:24.756 INFO:teuthology.orchestra.run.smithi090.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2021-02-20T07:56:24.756 INFO:teuthology.orchestra.run.smithi090.stdout:Recommended packages: 2021-02-20T07:56:24.756 INFO:teuthology.orchestra.run.smithi090.stdout: thermald 2021-02-20T07:56:24.790 INFO:teuthology.orchestra.run.smithi090.stdout:The following NEW packages will be installed: 2021-02-20T07:56:24.790 INFO:teuthology.orchestra.run.smithi090.stdout: linux-headers-4.15.0-135 linux-headers-4.15.0-135-generic 2021-02-20T07:56:24.791 INFO:teuthology.orchestra.run.smithi090.stdout: linux-image-4.15.0-135-generic linux-modules-4.15.0-135-generic 2021-02-20T07:56:24.792 INFO:teuthology.orchestra.run.smithi090.stdout: linux-modules-extra-4.15.0-135-generic 2021-02-20T07:56:24.792 INFO:teuthology.orchestra.run.smithi090.stdout:The following packages will be upgraded: 2021-02-20T07:56:24.793 INFO:teuthology.orchestra.run.smithi090.stdout: linux-generic linux-headers-generic linux-image-generic 2021-02-20T07:56:24.969 INFO:teuthology.orchestra.run.smithi090.stdout:3 upgraded, 5 newly installed, 0 to remove and 38 not upgraded. 2021-02-20T07:56:24.970 INFO:teuthology.orchestra.run.smithi090.stdout:Need to get 66.0 MB of archives. 2021-02-20T07:56:24.970 INFO:teuthology.orchestra.run.smithi090.stdout:After this operation, 334 MB of additional disk space will be used. 2021-02-20T07:56:24.970 INFO:teuthology.orchestra.run.smithi090.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-135-generic amd64 4.15.0-135.139 [13.1 MB] 2021-02-20T07:56:25.936 INFO:teuthology.orchestra.run.smithi090.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-135-generic amd64 4.15.0-135.139 [8,019 kB] 2021-02-20T07:56:26.174 INFO:teuthology.orchestra.run.smithi090.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-135-generic amd64 4.15.0-135.139 [32.8 MB] 2021-02-20T07:56:27.121 INFO:teuthology.orchestra.run.smithi090.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.135.122 [1,868 B] 2021-02-20T07:56:27.121 INFO:teuthology.orchestra.run.smithi090.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.135.122 [2,592 B] 2021-02-20T07:56:27.122 INFO:teuthology.orchestra.run.smithi090.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-135 all 4.15.0-135.139 [10.9 MB] 2021-02-20T07:56:27.428 INFO:teuthology.orchestra.run.smithi090.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-135-generic amd64 4.15.0-135.139 [1,097 kB] 2021-02-20T07:56:27.464 INFO:teuthology.orchestra.run.smithi090.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.135.122 [2,488 B] 2021-02-20T07:56:27.950 INFO:teuthology.orchestra.run.smithi090.stdout:Fetched 66.0 MB in 3s (24.8 MB/s) 2021-02-20T07:56:28.102 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package linux-modules-4.15.0-135-generic. 2021-02-20T07:56:28.141 INFO:teuthology.orchestra.run.smithi090.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102079 files and directories currently installed.) 2021-02-20T07:56:28.143 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../0-linux-modules-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T07:56:28.194 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking linux-modules-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:56:29.791 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package linux-image-4.15.0-135-generic. 2021-02-20T07:56:29.805 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../1-linux-image-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T07:56:29.913 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking linux-image-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:56:30.413 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-135-generic. 2021-02-20T07:56:30.426 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T07:56:30.463 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking linux-modules-extra-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:56:33.780 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../3-linux-generic_4.15.0.135.122_amd64.deb ... 2021-02-20T07:56:33.908 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking linux-generic (4.15.0.135.122) over (4.15.0.134.121) ... 2021-02-20T07:56:34.219 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.135.122_amd64.deb ... 2021-02-20T07:56:34.347 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking linux-image-generic (4.15.0.135.122) over (4.15.0.134.121) ... 2021-02-20T07:56:34.717 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package linux-headers-4.15.0-135. 2021-02-20T07:56:34.730 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../5-linux-headers-4.15.0-135_4.15.0-135.139_all.deb ... 2021-02-20T07:56:34.775 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking linux-headers-4.15.0-135 (4.15.0-135.139) ... 2021-02-20T07:56:38.593 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package linux-headers-4.15.0-135-generic. 2021-02-20T07:56:38.615 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../6-linux-headers-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T07:56:38.651 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking linux-headers-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:56:40.605 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.135.122_amd64.deb ... 2021-02-20T07:56:40.729 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking linux-headers-generic (4.15.0.135.122) over (4.15.0.134.121) ... 2021-02-20T07:56:41.064 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up linux-modules-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:56:41.621 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up linux-headers-4.15.0-135 (4.15.0-135.139) ... 2021-02-20T07:56:41.747 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up linux-image-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:56:42.202 INFO:teuthology.orchestra.run.smithi090.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-135-generic 2021-02-20T07:56:42.202 INFO:teuthology.orchestra.run.smithi090.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-135-generic 2021-02-20T07:56:42.341 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up linux-modules-extra-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:56:42.828 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up linux-headers-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:56:42.954 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up linux-image-generic (4.15.0.135.122) ... 2021-02-20T07:56:43.080 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up linux-headers-generic (4.15.0.135.122) ... 2021-02-20T07:56:43.206 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up linux-generic (4.15.0.135.122) ... 2021-02-20T07:56:43.333 INFO:teuthology.orchestra.run.smithi090.stdout:Processing triggers for linux-image-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:56:43.538 INFO:teuthology.orchestra.run.smithi090.stdout:/etc/kernel/postinst.d/initramfs-tools: 2021-02-20T07:56:43.539 INFO:teuthology.orchestra.run.smithi090.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-135-generic 2021-02-20T07:56:58.900 INFO:teuthology.orchestra.run.smithi090.stdout:/etc/kernel/postinst.d/zz-update-grub: 2021-02-20T07:56:58.901 INFO:teuthology.orchestra.run.smithi090.stdout:Sourcing file `/etc/default/grub' 2021-02-20T07:56:58.938 INFO:teuthology.orchestra.run.smithi090.stdout:Generating grub configuration file ... 2021-02-20T07:56:59.614 INFO:teuthology.orchestra.run.smithi090.stdout:Found linux image: /boot/vmlinuz-4.15.0-135-generic 2021-02-20T07:56:59.666 INFO:teuthology.orchestra.run.smithi090.stdout:Found initrd image: /boot/initrd.img-4.15.0-135-generic 2021-02-20T07:57:00.118 INFO:teuthology.orchestra.run.smithi090.stdout:Found linux image: /boot/vmlinuz-4.15.0-134-generic 2021-02-20T07:57:00.157 INFO:teuthology.orchestra.run.smithi090.stdout:Found initrd image: /boot/initrd.img-4.15.0-134-generic 2021-02-20T07:57:01.353 INFO:teuthology.orchestra.run.smithi090.stdout:done 2021-02-20T07:57:01.792 DEBUG:teuthology.orchestra.run.smithi090:> dpkg -s linux-image-generic 2021-02-20T07:57:01.816 INFO:teuthology.orchestra.run.smithi090.stdout:Package: linux-image-generic 2021-02-20T07:57:01.816 INFO:teuthology.orchestra.run.smithi090.stdout:Status: install ok installed 2021-02-20T07:57:01.817 INFO:teuthology.orchestra.run.smithi090.stdout:Priority: optional 2021-02-20T07:57:01.817 INFO:teuthology.orchestra.run.smithi090.stdout:Section: kernel 2021-02-20T07:57:01.817 INFO:teuthology.orchestra.run.smithi090.stdout:Installed-Size: 17 2021-02-20T07:57:01.817 INFO:teuthology.orchestra.run.smithi090.stdout:Maintainer: Ubuntu Kernel Team 2021-02-20T07:57:01.818 INFO:teuthology.orchestra.run.smithi090.stdout:Architecture: amd64 2021-02-20T07:57:01.818 INFO:teuthology.orchestra.run.smithi090.stdout:Source: linux-meta 2021-02-20T07:57:01.818 INFO:teuthology.orchestra.run.smithi090.stdout:Version: 4.15.0.135.122 2021-02-20T07:57:01.818 INFO:teuthology.orchestra.run.smithi090.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-02-20T07:57:01.818 INFO:teuthology.orchestra.run.smithi090.stdout:Depends: linux-image-4.15.0-135-generic, linux-modules-extra-4.15.0-135-generic, linux-firmware, intel-microcode, amd64-microcode 2021-02-20T07:57:01.819 INFO:teuthology.orchestra.run.smithi090.stdout:Recommends: thermald 2021-02-20T07:57:01.821 INFO:teuthology.orchestra.run.smithi090.stdout:Description: Generic Linux kernel image 2021-02-20T07:57:01.821 INFO:teuthology.orchestra.run.smithi090.stdout: This package will always depend on the latest generic kernel image 2021-02-20T07:57:01.822 INFO:teuthology.orchestra.run.smithi090.stdout: available. 2021-02-20T07:57:01.822 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-134-generic Expected: 4.15.0-135-generic 2021-02-20T07:57:01.823 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2021-02-20T07:57:01.823 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-02-20T07:57:01.823 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-02-20T07:57:01.824 INFO:teuthology.task.kernel:Installing distro kernel on cluster1.mon.a... 2021-02-20T07:57:01.824 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi003.front.sepia.ceph.com, path=None, version=distro) 2021-02-20T07:57:01.825 DEBUG:teuthology.orchestra.run.smithi003:> sudo apt-get clean 2021-02-20T07:57:01.867 DEBUG:teuthology.orchestra.run.smithi003:> sudo apt-get update 2021-02-20T07:57:02.253 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-02-20T07:57:02.320 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-02-20T07:57:02.398 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-02-20T07:57:02.421 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:4 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-02-20T07:57:04.221 INFO:teuthology.orchestra.run.smithi003.stdout:Reading package lists... 2021-02-20T07:57:04.268 DEBUG:teuthology.orchestra.run.smithi003:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-02-20T07:57:04.356 INFO:teuthology.orchestra.run.smithi003.stdout:Reading package lists... 2021-02-20T07:57:04.606 INFO:teuthology.orchestra.run.smithi003.stdout:Building dependency tree... 2021-02-20T07:57:04.607 INFO:teuthology.orchestra.run.smithi003.stdout:Reading state information... 2021-02-20T07:57:04.693 INFO:teuthology.orchestra.run.smithi003.stderr:E: Unable to locate package linux-image-current-generic 2021-02-20T07:57:04.694 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-02-20T07:57:04.695 DEBUG:teuthology.orchestra.run.smithi003:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-02-20T07:57:04.778 INFO:teuthology.orchestra.run.smithi003.stdout:Reading package lists... 2021-02-20T07:57:05.035 INFO:teuthology.orchestra.run.smithi003.stdout:Building dependency tree... 2021-02-20T07:57:05.036 INFO:teuthology.orchestra.run.smithi003.stdout:Reading state information... 2021-02-20T07:57:05.250 INFO:teuthology.orchestra.run.smithi003.stdout:linux-image-generic is already the newest version (4.15.0.135.122). 2021-02-20T07:57:05.251 INFO:teuthology.orchestra.run.smithi003.stdout:0 upgraded, 0 newly installed, 0 to remove and 38 not upgraded. 2021-02-20T07:57:05.252 DEBUG:teuthology.orchestra.run.smithi003:> dpkg -s linux-image-generic 2021-02-20T07:57:05.321 INFO:teuthology.orchestra.run.smithi003.stdout:Package: linux-image-generic 2021-02-20T07:57:05.321 INFO:teuthology.orchestra.run.smithi003.stdout:Status: install ok installed 2021-02-20T07:57:05.322 INFO:teuthology.orchestra.run.smithi003.stdout:Priority: optional 2021-02-20T07:57:05.322 INFO:teuthology.orchestra.run.smithi003.stdout:Section: kernel 2021-02-20T07:57:05.322 INFO:teuthology.orchestra.run.smithi003.stdout:Installed-Size: 17 2021-02-20T07:57:05.322 INFO:teuthology.orchestra.run.smithi003.stdout:Maintainer: Ubuntu Kernel Team 2021-02-20T07:57:05.323 INFO:teuthology.orchestra.run.smithi003.stdout:Architecture: amd64 2021-02-20T07:57:05.323 INFO:teuthology.orchestra.run.smithi003.stdout:Source: linux-meta 2021-02-20T07:57:05.323 INFO:teuthology.orchestra.run.smithi003.stdout:Version: 4.15.0.135.122 2021-02-20T07:57:05.323 INFO:teuthology.orchestra.run.smithi003.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-02-20T07:57:05.323 INFO:teuthology.orchestra.run.smithi003.stdout:Depends: linux-image-4.15.0-135-generic, linux-modules-extra-4.15.0-135-generic, linux-firmware, intel-microcode, amd64-microcode 2021-02-20T07:57:05.324 INFO:teuthology.orchestra.run.smithi003.stdout:Recommends: thermald 2021-02-20T07:57:05.324 INFO:teuthology.orchestra.run.smithi003.stdout:Description: Generic Linux kernel image 2021-02-20T07:57:05.324 INFO:teuthology.orchestra.run.smithi003.stdout: This package will always depend on the latest generic kernel image 2021-02-20T07:57:05.324 INFO:teuthology.orchestra.run.smithi003.stdout: available. 2021-02-20T07:57:05.325 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2021-02-20T07:57:05.371 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.tqeddb003X 2021-02-20T07:57:05.372 DEBUG:teuthology.orchestra.run.smithi003:> sudo cp /boot/grub/grub.cfg /tmp/tmp.tqeddb003X 2021-02-20T07:57:05.428 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.tqeddb003X 2021-02-20T07:57:05.545 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.tqeddb003X is 8KB 2021-02-20T07:57:05.559 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.tqeddb003X 2021-02-20T07:57:05.567 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2021-02-20T07:57:05.622 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T07:57:05.623 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/grub.d/01_ceph_kernel 2021-02-20T07:57:05.623 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2021-02-20T07:57:05.704 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-135-generic 2021-02-20T07:57:05.705 DEBUG:teuthology.orchestra.run.smithi003:> sudo update-grub 2021-02-20T07:57:06.972 INFO:teuthology.orchestra.run.smithi003.stderr:Sourcing file `/etc/default/grub' 2021-02-20T07:57:06.998 INFO:teuthology.orchestra.run.smithi003.stderr:Generating grub configuration file ... 2021-02-20T07:57:07.220 INFO:teuthology.orchestra.run.smithi003.stderr:Found linux image: /boot/vmlinuz-4.15.0-135-generic 2021-02-20T07:57:07.232 INFO:teuthology.orchestra.run.smithi003.stderr:Found initrd image: /boot/initrd.img-4.15.0-135-generic 2021-02-20T07:57:07.509 INFO:teuthology.orchestra.run.smithi003.stderr:Found linux image: /boot/vmlinuz-4.15.0-134-generic 2021-02-20T07:57:07.522 INFO:teuthology.orchestra.run.smithi003.stderr:Found initrd image: /boot/initrd.img-4.15.0-134-generic 2021-02-20T07:57:08.127 INFO:teuthology.orchestra.run.smithi003.stderr:done 2021-02-20T07:57:08.128 DEBUG:teuthology.orchestra.run.smithi003:> sudo shutdown -r now 2021-02-20T07:57:08.131 INFO:teuthology.task.kernel:Installing distro kernel on cluster2.mon.a... 2021-02-20T07:57:08.131 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi090.front.sepia.ceph.com, path=None, version=distro) 2021-02-20T07:57:08.131 DEBUG:teuthology.orchestra.run.smithi090:> sudo apt-get clean 2021-02-20T07:57:08.210 DEBUG:teuthology.orchestra.run.smithi090:> sudo apt-get update 2021-02-20T07:57:08.453 INFO:teuthology.orchestra.run.smithi090.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-02-20T07:57:08.454 INFO:teuthology.orchestra.run.smithi090.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-02-20T07:57:08.523 INFO:teuthology.orchestra.run.smithi090.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-02-20T07:57:08.603 INFO:teuthology.orchestra.run.smithi090.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-02-20T07:57:10.341 INFO:teuthology.orchestra.run.smithi090.stdout:Reading package lists... 2021-02-20T07:57:10.387 DEBUG:teuthology.orchestra.run.smithi090:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-02-20T07:57:10.460 INFO:teuthology.orchestra.run.smithi090.stdout:Reading package lists... 2021-02-20T07:57:10.693 INFO:teuthology.orchestra.run.smithi090.stdout:Building dependency tree... 2021-02-20T07:57:10.694 INFO:teuthology.orchestra.run.smithi090.stdout:Reading state information... 2021-02-20T07:57:10.774 INFO:teuthology.orchestra.run.smithi090.stderr:E: Unable to locate package linux-image-current-generic 2021-02-20T07:57:10.775 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-02-20T07:57:10.776 DEBUG:teuthology.orchestra.run.smithi090:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-02-20T07:57:10.896 INFO:teuthology.orchestra.run.smithi090.stdout:Reading package lists... 2021-02-20T07:57:11.124 INFO:teuthology.orchestra.run.smithi090.stdout:Building dependency tree... 2021-02-20T07:57:11.125 INFO:teuthology.orchestra.run.smithi090.stdout:Reading state information... 2021-02-20T07:57:11.322 INFO:teuthology.orchestra.run.smithi090.stdout:linux-image-generic is already the newest version (4.15.0.135.122). 2021-02-20T07:57:11.322 INFO:teuthology.orchestra.run.smithi090.stdout:0 upgraded, 0 newly installed, 0 to remove and 38 not upgraded. 2021-02-20T07:57:11.324 DEBUG:teuthology.orchestra.run.smithi090:> dpkg -s linux-image-generic 2021-02-20T07:57:11.341 INFO:teuthology.orchestra.run.smithi090.stdout:Package: linux-image-generic 2021-02-20T07:57:11.341 INFO:teuthology.orchestra.run.smithi090.stdout:Status: install ok installed 2021-02-20T07:57:11.342 INFO:teuthology.orchestra.run.smithi090.stdout:Priority: optional 2021-02-20T07:57:11.342 INFO:teuthology.orchestra.run.smithi090.stdout:Section: kernel 2021-02-20T07:57:11.343 INFO:teuthology.orchestra.run.smithi090.stdout:Installed-Size: 17 2021-02-20T07:57:11.343 INFO:teuthology.orchestra.run.smithi090.stdout:Maintainer: Ubuntu Kernel Team 2021-02-20T07:57:11.343 INFO:teuthology.orchestra.run.smithi090.stdout:Architecture: amd64 2021-02-20T07:57:11.344 INFO:teuthology.orchestra.run.smithi090.stdout:Source: linux-meta 2021-02-20T07:57:11.344 INFO:teuthology.orchestra.run.smithi090.stdout:Version: 4.15.0.135.122 2021-02-20T07:57:11.344 INFO:teuthology.orchestra.run.smithi090.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-02-20T07:57:11.344 INFO:teuthology.orchestra.run.smithi090.stdout:Depends: linux-image-4.15.0-135-generic, linux-modules-extra-4.15.0-135-generic, linux-firmware, intel-microcode, amd64-microcode 2021-02-20T07:57:11.344 INFO:teuthology.orchestra.run.smithi090.stdout:Recommends: thermald 2021-02-20T07:57:11.345 INFO:teuthology.orchestra.run.smithi090.stdout:Description: Generic Linux kernel image 2021-02-20T07:57:11.345 INFO:teuthology.orchestra.run.smithi090.stdout: This package will always depend on the latest generic kernel image 2021-02-20T07:57:11.345 INFO:teuthology.orchestra.run.smithi090.stdout: available. 2021-02-20T07:57:11.346 DEBUG:teuthology.orchestra.run.smithi090:> mktemp 2021-02-20T07:57:11.388 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/tmp.3GG0JItxqs 2021-02-20T07:57:11.389 DEBUG:teuthology.orchestra.run.smithi090:> sudo cp /boot/grub/grub.cfg /tmp/tmp.3GG0JItxqs 2021-02-20T07:57:11.446 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0666 /tmp/tmp.3GG0JItxqs 2021-02-20T07:57:11.564 DEBUG:teuthology.orchestra.remote:smithi090:/tmp/tmp.3GG0JItxqs is 8KB 2021-02-20T07:57:11.613 DEBUG:teuthology.orchestra.run.smithi090:> rm -fr /tmp/tmp.3GG0JItxqs 2021-02-20T07:57:11.619 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2021-02-20T07:57:11.673 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T07:57:11.674 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/grub.d/01_ceph_kernel 2021-02-20T07:57:11.674 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2021-02-20T07:57:11.750 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-135-generic 2021-02-20T07:57:11.750 DEBUG:teuthology.orchestra.run.smithi090:> sudo update-grub 2021-02-20T07:57:13.426 INFO:teuthology.orchestra.run.smithi090.stderr:Sourcing file `/etc/default/grub' 2021-02-20T07:57:13.457 INFO:teuthology.orchestra.run.smithi090.stderr:Generating grub configuration file ... 2021-02-20T07:57:13.796 INFO:teuthology.orchestra.run.smithi090.stderr:Found linux image: /boot/vmlinuz-4.15.0-135-generic 2021-02-20T07:57:13.808 INFO:teuthology.orchestra.run.smithi090.stderr:Found initrd image: /boot/initrd.img-4.15.0-135-generic 2021-02-20T07:57:14.306 INFO:teuthology.orchestra.run.smithi090.stderr:Found linux image: /boot/vmlinuz-4.15.0-134-generic 2021-02-20T07:57:14.319 INFO:teuthology.orchestra.run.smithi090.stderr:Found initrd image: /boot/initrd.img-4.15.0-134-generic 2021-02-20T07:57:14.922 INFO:teuthology.orchestra.run.smithi090.stderr:done 2021-02-20T07:57:14.923 DEBUG:teuthology.orchestra.run.smithi090:> sudo shutdown -r now 2021-02-20T07:57:44.964 INFO:teuthology.misc:Re-opening connections... 2021-02-20T07:57:44.965 INFO:teuthology.misc:trying to connect to ubuntu@smithi003.front.sepia.ceph.com 2021-02-20T07:57:44.965 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-20T07:57:44.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T07:58:03.462 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.3 2021-02-20T07:58:03.471 INFO:teuthology.misc:trying to connect to ubuntu@smithi090.front.sepia.ceph.com 2021-02-20T07:58:03.471 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-20T07:58:03.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T07:58:21.894 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2021-02-20T07:58:21.895 DEBUG:teuthology.misc:waited 36.93030071258545 2021-02-20T07:58:22.896 INFO:teuthology.misc:trying to connect to ubuntu@smithi003.front.sepia.ceph.com 2021-02-20T07:58:22.897 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-20T07:58:22.897 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T07:58:38.477 DEBUG:teuthology.orchestra.run.smithi003:> true 2021-02-20T07:58:39.821 DEBUG:teuthology.misc:waited 54.85627341270447 2021-02-20T07:58:40.823 INFO:teuthology.misc:trying to connect to ubuntu@smithi090.front.sepia.ceph.com 2021-02-20T07:58:40.824 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-20T07:58:40.824 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T07:59:40.828 DEBUG:teuthology.orchestra.remote:timed out 2021-02-20T07:59:40.832 DEBUG:teuthology.misc:waited 115.86723446846008 2021-02-20T07:59:41.833 INFO:teuthology.misc:trying to connect to ubuntu@smithi090.front.sepia.ceph.com 2021-02-20T07:59:41.834 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-20T07:59:41.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T07:59:41.916 DEBUG:teuthology.orchestra.run.smithi090:> true 2021-02-20T07:59:42.373 DEBUG:teuthology.misc:waited 117.40755605697632 2021-02-20T07:59:43.374 INFO:teuthology.task.kernel:Checking client cluster1.mon.a for new kernel version... 2021-02-20T07:59:43.374 INFO:teuthology.task.kernel:Checking kernel version of cluster1.mon.a, want "4.15.0-135-generic"... 2021-02-20T07:59:43.375 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2021-02-20T07:59:43.381 INFO:teuthology.orchestra.run.smithi003.stdout:4.15.0-135-generic 2021-02-20T07:59:43.382 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-135-generic vs 4.15.0-135-generic 2021-02-20T07:59:43.382 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-02-20T07:59:43.383 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2021-02-20T07:59:43.383 INFO:teuthology.task.kernel:Checking client cluster2.mon.a for new kernel version... 2021-02-20T07:59:43.383 INFO:teuthology.task.kernel:Checking kernel version of cluster2.mon.a, want "4.15.0-135-generic"... 2021-02-20T07:59:43.383 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2021-02-20T07:59:43.388 INFO:teuthology.orchestra.run.smithi090.stdout:4.15.0-135-generic 2021-02-20T07:59:43.389 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-135-generic vs 4.15.0-135-generic 2021-02-20T07:59:43.389 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-02-20T07:59:43.389 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2021-02-20T07:59:44.390 INFO:teuthology.task.kernel:Enabling kdb on cluster1.mon.a... 2021-02-20T07:59:44.391 DEBUG:teuthology.orchestra.run.smithi003:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-02-20T07:59:44.421 INFO:teuthology.orchestra.run.smithi003.stdout:ttyS1 2021-02-20T07:59:44.449 INFO:teuthology.task.kernel:Enabling kdb on cluster2.mon.a... 2021-02-20T07:59:44.449 DEBUG:teuthology.orchestra.run.smithi090:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-02-20T07:59:44.501 INFO:teuthology.orchestra.run.smithi090.stdout:ttyS1 2021-02-20T07:59:44.526 INFO:teuthology.run_tasks:Running task internal.base... 2021-02-20T07:59:44.542 INFO:teuthology.task.internal:Creating test directory... 2021-02-20T07:59:44.543 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-02-20T07:59:44.546 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-02-20T07:59:44.553 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2021-02-20T07:59:44.584 INFO:teuthology.run_tasks:Running task internal.archive... 2021-02-20T07:59:44.606 INFO:teuthology.task.internal:Creating archive directory... 2021-02-20T07:59:44.606 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-02-20T07:59:44.609 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-02-20T07:59:44.674 INFO:teuthology.run_tasks:Running task internal.coredump... 2021-02-20T07:59:44.687 INFO:teuthology.task.internal:Enabling coredump saving... 2021-02-20T07:59:44.688 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-02-20T07:59:44.691 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-02-20T07:59:44.712 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-02-20T07:59:44.712 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-02-20T07:59:44.719 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-02-20T07:59:44.723 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-02-20T07:59:44.725 INFO:teuthology.run_tasks:Running task internal.sudo... 2021-02-20T07:59:44.742 INFO:teuthology.task.internal:Configuring sudo... 2021-02-20T07:59:44.743 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-02-20T07:59:44.768 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-02-20T07:59:44.784 INFO:teuthology.run_tasks:Running task internal.syslog... 2021-02-20T07:59:44.797 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2021-02-20T07:59:44.798 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-02-20T07:59:44.825 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-02-20T07:59:44.833 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-02-20T07:59:44.876 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-02-20T07:59:44.925 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T07:59:44.926 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-02-20T07:59:44.987 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-02-20T07:59:44.991 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-02-20T07:59:45.035 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T07:59:45.036 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-02-20T07:59:45.100 DEBUG:teuthology.orchestra.run.smithi003:> sudo service rsyslog restart 2021-02-20T07:59:45.103 DEBUG:teuthology.orchestra.run.smithi090:> sudo service rsyslog restart 2021-02-20T07:59:45.189 INFO:teuthology.run_tasks:Running task internal.timer... 2021-02-20T07:59:45.198 INFO:teuthology.task.internal:Starting timer... 2021-02-20T07:59:45.198 INFO:teuthology.run_tasks:Running task pcp... 2021-02-20T07:59:45.217 INFO:teuthology.run_tasks:Running task selinux... 2021-02-20T07:59:45.235 DEBUG:teuthology.task.selinux:Excluding smithi003: OS 'ubuntu' does not support SELinux 2021-02-20T07:59:45.236 DEBUG:teuthology.task.selinux:Excluding smithi090: OS 'ubuntu' does not support SELinux 2021-02-20T07:59:45.236 DEBUG:teuthology.task.selinux:Getting current SELinux state 2021-02-20T07:59:45.236 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2021-02-20T07:59:45.237 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2021-02-20T07:59:45.237 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2021-02-20T07:59:45.252 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated or references a specific commit; assuming it is current 2021-02-20T07:59:45.252 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to origin/master 2021-02-20T07:59:45.510 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2021-02-20T07:59:45.515 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi003.front.sepia.ceph.com,smithi090.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2021-02-20T07:59:46.376 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2021-02-20T07:59:48.279 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2021-02-20T07:59:51.826 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2021-02-20T07:59:51.865 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:51 +0000 (0:00:00.168) 0:00:00.168 ***** 2021-02-20T07:59:51.887 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:51 +0000 (0:00:00.022) 0:00:00.191 ***** 2021-02-20T07:59:54.070 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi003.front.sepia.ceph.com] 2021-02-20T07:59:54.120 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] 2021-02-20T07:59:54.169 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:54 +0000 (0:00:02.281) 0:00:02.472 ***** 2021-02-20T07:59:54.242 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:54 +0000 (0:00:00.073) 0:00:02.546 ***** 2021-02-20T07:59:54.505 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2021-02-20T07:59:54.507 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-02-20T07:59:54.549 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:54 +0000 (0:00:00.306) 0:00:02.853 ***** 2021-02-20T07:59:54.557 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-02-20T07:59:54.597 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:54 +0000 (0:00:00.047) 0:00:02.900 ***** 2021-02-20T07:59:55.000 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** changed: [smithi090.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-02-20T07:59:55.036 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:55 +0000 (0:00:00.438) 0:00:03.339 ***** 2021-02-20T07:59:55.042 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-02-20T07:59:55.081 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:55 +0000 (0:00:00.045) 0:00:03.384 ***** 2021-02-20T07:59:55.364 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2021-02-20T07:59:55.365 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.062489", "end": "2021-02-20 07:59:55.477524", "rc": 0, "start": "2021-02-20 07:59:55.415035", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2021-02-20T07:59:55.403 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:55 +0000 (0:00:00.321) 0:00:03.706 ***** 2021-02-20T07:59:55.497 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.199346", "end": "2021-02-20 07:59:55.594475", "rc": 0, "start": "2021-02-20 07:59:55.395129", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2021-02-20T07:59:55.535 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:55 +0000 (0:00:00.132) 0:00:03.839 ***** 2021-02-20T07:59:55.667 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** ok: [smithi090.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T07:59:55.709 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:55 +0000 (0:00:00.173) 0:00:04.012 ***** 2021-02-20T07:59:55.716 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T07:59:55.758 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:55 +0000 (0:00:00.049) 0:00:04.062 ***** 2021-02-20T07:59:56.837 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** changed: [smithi003.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613807995.7748702-195869569063475/source", "state": "file", "uid": 0} 2021-02-20T07:59:56.876 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:56 +0000 (0:00:01.114) 0:00:05.176 ***** 2021-02-20T07:59:56.880 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613807995.7278957-61335783354942/source", "state": "file", "uid": 0} 2021-02-20T07:59:56.946 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:56 +0000 (0:00:00.071) 0:00:05.248 ***** 2021-02-20T07:59:57.331 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2021-02-20T07:59:57.336 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2021-02-20T07:59:57.337 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-02-20T07:59:57.367 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2021-02-20T07:59:57.369 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-02-20T07:59:57.513 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2021-02-20T07:59:57.521 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2021-02-20T07:59:57.677 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:57 +0000 (0:00:00.732) 0:00:05.981 ***** 2021-02-20T07:59:57.715 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:57 +0000 (0:00:00.036) 0:00:06.017 ***** 2021-02-20T07:59:58.449 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2021-02-20T07:59:58.452 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] 2021-02-20T07:59:58.542 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. 2021-02-20T07:59:58.544 INFO:teuthology.task.ansible.out: This task will still be executed for every host in the inventory list. Saturday 20 February 2021 07:59:58 +0000 (0:00:00.829) 0:00:06.847 ***** 2021-02-20T07:59:58.557 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] 2021-02-20T07:59:58.666 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:58 +0000 (0:00:00.122) 0:00:06.969 ***** 2021-02-20T07:59:59.155 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2021-02-20T07:59:59.157 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.295180", "end": "2021-02-20 07:59:59.129854", "rc": 0, "start": "2021-02-20 07:59:58.834674", "stderr": "", "stderr_lines": [], "stdout": "df8946f08e870e2d2565b79099289ca48bd825f3", "stdout_lines": ["df8946f08e870e2d2565b79099289ca48bd825f3"]} 2021-02-20T07:59:59.199 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:59 +0000 (0:00:00.533) 0:00:07.503 ***** 2021-02-20T07:59:59.206 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.262459", "end": "2021-02-20 07:59:59.124714", "rc": 0, "start": "2021-02-20 07:59:58.862255", "stderr": "", "stderr_lines": [], "stdout": "df8946f08e870e2d2565b79099289ca48bd825f3", "stdout_lines": ["df8946f08e870e2d2565b79099289ca48bd825f3"]} 2021-02-20T07:59:59.251 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:59 +0000 (0:00:00.051) 0:00:07.554 ***** 2021-02-20T07:59:59.345 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2021-02-20T07:59:59.346 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.010482", "end": "2021-02-20 07:59:59.433958", "failed_when_result": false, "rc": 0, "start": "2021-02-20 07:59:59.423476", "stderr": "", "stderr_lines": [], "stdout": "0caff10a478f2674721da1965013b225f3ea5800", "stdout_lines": ["0caff10a478f2674721da1965013b225f3ea5800"]} 2021-02-20T07:59:59.409 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:59 +0000 (0:00:00.158) 0:00:07.712 ***** 2021-02-20T07:59:59.417 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.014555", "end": "2021-02-20 07:59:59.500502", "failed_when_result": false, "rc": 0, "start": "2021-02-20 07:59:59.485947", "stderr": "", "stderr_lines": [], "stdout": "0caff10a478f2674721da1965013b225f3ea5800", "stdout_lines": ["0caff10a478f2674721da1965013b225f3ea5800"]} 2021-02-20T07:59:59.458 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:59 +0000 (0:00:00.049) 0:00:07.762 ***** 2021-02-20T07:59:59.465 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2021-02-20T07:59:59.466 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:59:59.513 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:59 +0000 (0:00:00.054) 0:00:07.817 ***** 2021-02-20T07:59:59.528 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:59:59.569 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:59 +0000 (0:00:00.055) 0:00:07.872 ***** 2021-02-20T07:59:59.576 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2021-02-20T07:59:59.578 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:59:59.619 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:59 +0000 (0:00:00.049) 0:00:07.922 ***** 2021-02-20T07:59:59.626 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:59:59.672 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:59 +0000 (0:00:00.053) 0:00:07.976 ***** 2021-02-20T07:59:59.679 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2021-02-20T07:59:59.681 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:59:59.734 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:59 +0000 (0:00:00.061) 0:00:08.038 ***** 2021-02-20T07:59:59.747 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:59:59.795 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:59 +0000 (0:00:00.060) 0:00:08.098 ***** 2021-02-20T07:59:59.805 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2021-02-20T07:59:59.807 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:59:59.842 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:59 +0000 (0:00:00.047) 0:00:08.146 ***** 2021-02-20T07:59:59.849 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:59:59.890 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:59 +0000 (0:00:00.047) 0:00:08.194 ***** 2021-02-20T07:59:59.897 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2021-02-20T07:59:59.898 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:59:59.934 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:59 +0000 (0:00:00.043) 0:00:08.238 ***** 2021-02-20T07:59:59.943 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:59:59.980 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:59:59 +0000 (0:00:00.046) 0:00:08.284 ***** 2021-02-20T08:00:00.081 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2021-02-20T08:00:00.083 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T08:00:00.119 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:00:00 +0000 (0:00:00.139) 0:00:08.423 ***** 2021-02-20T08:00:00.126 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T08:00:00.169 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:00:00 +0000 (0:00:00.048) 0:00:08.472 ***** 2021-02-20T08:00:00.337 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-02-20T08:00:00.348 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-02-20T08:00:00.478 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-02-20T08:00:00.489 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-02-20T08:00:00.609 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-02-20T08:00:00.611 INFO:teuthology.task.ansible.out: 2021-02-20T08:00:00.632 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-02-20T08:00:00.756 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-02-20T08:00:00.779 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-02-20T08:00:00.907 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-02-20T08:00:00.919 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-02-20T08:00:01.058 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-02-20T08:00:01.073 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-02-20T08:00:01.207 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-02-20T08:00:01.218 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-02-20T08:00:01.336 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-02-20T08:00:01.368 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-02-20T08:00:01.458 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-02-20T08:00:01.502 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-02-20T08:00:01.598 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-02-20T08:00:01.652 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-02-20T08:00:01.737 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-02-20T08:00:01.791 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-02-20T08:00:01.865 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-02-20T08:00:01.918 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-02-20T08:00:02.016 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-02-20T08:00:02.049 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-02-20T08:00:02.167 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-02-20T08:00:02.208 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-02-20T08:00:02.321 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-02-20T08:00:02.386 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-02-20T08:00:02.493 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-02-20T08:00:02.551 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-02-20T08:00:02.652 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-02-20T08:00:02.706 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-02-20T08:00:02.707 INFO:teuthology.task.ansible.out: 2021-02-20T08:00:02.806 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-02-20T08:00:02.893 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-02-20T08:00:03.028 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-02-20T08:00:03.110 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-02-20T08:00:03.112 INFO:teuthology.task.ansible.out: 2021-02-20T08:00:03.208 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-02-20T08:00:03.256 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-02-20T08:00:03.408 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-02-20T08:00:03.430 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-02-20T08:00:03.581 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-02-20T08:00:03.586 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-02-20T08:00:03.716 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-02-20T08:00:03.739 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-02-20T08:00:03.882 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-02-20T08:00:03.906 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-02-20T08:00:04.028 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-02-20T08:00:04.064 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-02-20T08:00:04.186 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-02-20T08:00:04.212 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-02-20T08:00:04.363 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-02-20T08:00:04.387 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-02-20T08:00:04.508 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-02-20T08:00:04.528 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-02-20T08:00:04.648 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-02-20T08:00:04.686 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-02-20T08:00:04.810 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-02-20T08:00:04.853 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-02-20T08:00:04.855 INFO:teuthology.task.ansible.out: 2021-02-20T08:00:04.985 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-02-20T08:00:05.018 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-02-20T08:00:05.146 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-02-20T08:00:05.177 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-02-20T08:00:05.304 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-02-20T08:00:05.341 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-02-20T08:00:05.463 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-02-20T08:00:05.485 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-02-20T08:00:05.634 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-02-20T08:00:05.646 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-02-20T08:00:05.782 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-02-20T08:00:05.783 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-02-20T08:00:05.939 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-02-20T08:00:05.962 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-02-20T08:00:06.095 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-02-20T08:00:06.096 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-02-20T08:00:06.239 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-02-20T08:00:06.240 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-02-20T08:00:06.391 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-02-20T08:00:06.403 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-02-20T08:00:06.544 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-02-20T08:00:06.559 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-02-20T08:00:06.703 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-02-20T08:00:06.712 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-02-20T08:00:06.855 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-02-20T08:00:06.864 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-02-20T08:00:07.018 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-02-20T08:00:07.041 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-02-20T08:00:07.155 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-02-20T08:00:07.210 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-02-20T08:00:07.308 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-02-20T08:00:07.393 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-02-20T08:00:07.459 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-02-20T08:00:07.536 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-02-20T08:00:07.590 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-02-20T08:00:07.698 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-02-20T08:00:07.730 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-02-20T08:00:07.873 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-02-20T08:00:07.885 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-02-20T08:00:08.034 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-02-20T08:00:08.046 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-02-20T08:00:08.176 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-02-20T08:00:08.198 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-02-20T08:00:08.325 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-02-20T08:00:08.359 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-02-20T08:00:08.475 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-02-20T08:00:08.518 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-02-20T08:00:08.594 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-02-20T08:00:08.662 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-02-20T08:00:08.728 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-02-20T08:00:08.815 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-02-20T08:00:08.871 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-02-20T08:00:08.939 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-02-20T08:00:08.993 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-02-20T08:00:09.090 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-02-20T08:00:09.158 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-02-20T08:00:09.215 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-02-20T08:00:09.283 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-02-20T08:00:09.347 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-02-20T08:00:09.434 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-02-20T08:00:09.489 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-02-20T08:00:09.575 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-02-20T08:00:09.630 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-02-20T08:00:09.726 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-02-20T08:00:09.783 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-02-20T08:00:09.870 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-02-20T08:00:09.945 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-02-20T08:00:10.009 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-02-20T08:00:10.118 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-02-20T08:00:10.153 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-02-20T08:00:10.261 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-02-20T08:00:10.295 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-02-20T08:00:10.392 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-02-20T08:00:10.458 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-02-20T08:00:10.525 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-02-20T08:00:10.630 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-02-20T08:00:10.668 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-02-20T08:00:10.788 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-02-20T08:00:10.818 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-02-20T08:00:10.987 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-02-20T08:00:11.040 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-02-20T08:00:11.042 INFO:teuthology.task.ansible.out: 2021-02-20T08:00:11.152 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-02-20T08:00:11.217 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-02-20T08:00:11.300 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-02-20T08:00:11.390 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-02-20T08:00:11.465 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-02-20T08:00:11.562 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-02-20T08:00:11.648 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-02-20T08:00:11.780 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-02-20T08:00:11.802 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-02-20T08:00:11.958 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-02-20T08:00:11.991 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-02-20T08:00:12.110 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-02-20T08:00:12.144 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-02-20T08:00:12.295 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-02-20T08:00:12.329 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-02-20T08:00:12.443 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-02-20T08:00:12.497 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-02-20T08:00:12.593 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-02-20T08:00:12.658 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-02-20T08:00:12.745 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-02-20T08:00:12.813 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-02-20T08:00:12.914 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-02-20T08:00:12.954 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-02-20T08:00:13.092 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-02-20T08:00:13.132 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-02-20T08:00:13.244 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-02-20T08:00:13.298 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-02-20T08:00:13.407 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-02-20T08:00:13.450 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-02-20T08:00:13.559 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-02-20T08:00:13.604 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-02-20T08:00:13.708 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-02-20T08:00:13.774 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-02-20T08:00:13.872 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-02-20T08:00:13.981 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-02-20T08:00:14.083 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-02-20T08:00:14.228 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-02-20T08:00:14.283 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-02-20T08:00:14.420 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-02-20T08:00:14.474 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-02-20T08:00:14.557 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-02-20T08:00:14.627 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-02-20T08:00:14.737 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-02-20T08:00:14.770 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-02-20T08:00:14.894 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-02-20T08:00:14.917 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-02-20T08:00:15.039 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-02-20T08:00:15.071 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-02-20T08:00:15.200 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-02-20T08:00:15.213 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-02-20T08:00:15.356 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-02-20T08:00:15.379 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-02-20T08:00:15.485 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-02-20T08:00:15.550 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-02-20T08:00:15.638 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-02-20T08:00:15.704 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-02-20T08:00:15.792 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-02-20T08:00:15.866 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-02-20T08:00:15.921 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-02-20T08:00:16.009 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-02-20T08:00:16.065 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-02-20T08:00:16.163 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-02-20T08:00:16.165 INFO:teuthology.task.ansible.out: 2021-02-20T08:00:16.220 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-02-20T08:00:16.314 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-02-20T08:00:16.379 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-02-20T08:00:16.443 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-02-20T08:00:16.508 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-02-20T08:00:16.573 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-02-20T08:00:16.644 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-02-20T08:00:16.731 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-02-20T08:00:16.774 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-02-20T08:00:16.892 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-02-20T08:00:16.940 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-02-20T08:00:17.048 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-02-20T08:00:17.091 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-02-20T08:00:17.093 INFO:teuthology.task.ansible.out: 2021-02-20T08:00:17.210 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-02-20T08:00:17.244 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-02-20T08:00:17.361 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-02-20T08:00:17.418 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-02-20T08:00:17.576 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-02-20T08:00:17.599 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-02-20T08:00:17.804 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-02-20T08:00:17.827 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-02-20T08:00:18.016 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-02-20T08:00:18.119 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-02-20T08:00:18.212 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-02-20T08:00:18.335 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-02-20T08:00:18.496 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-02-20T08:00:18.567 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-02-20T08:00:18.782 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-02-20T08:00:18.883 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-02-20T08:00:19.006 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-02-20T08:00:19.091 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-02-20T08:00:19.253 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-02-20T08:00:19.255 INFO:teuthology.task.ansible.out: 2021-02-20T08:00:19.265 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-02-20T08:00:19.402 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-02-20T08:00:19.448 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-02-20T08:00:19.548 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-02-20T08:00:19.613 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-02-20T08:00:19.689 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-02-20T08:00:19.743 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-02-20T08:00:19.850 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-02-20T08:00:19.895 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-02-20T08:00:19.992 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-02-20T08:00:20.048 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-02-20T08:00:20.157 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-02-20T08:00:20.190 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-02-20T08:00:20.310 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-02-20T08:00:20.328 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-02-20T08:00:20.491 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-02-20T08:00:20.493 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-02-20T08:00:20.651 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-02-20T08:00:20.653 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-02-20T08:00:20.804 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-02-20T08:00:20.816 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-02-20T08:00:20.956 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-02-20T08:00:20.958 INFO:teuthology.task.ansible.out: 2021-02-20T08:00:20.969 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-02-20T08:00:21.087 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-02-20T08:00:21.120 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-02-20T08:00:21.217 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-02-20T08:00:21.274 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-02-20T08:00:21.350 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-02-20T08:00:21.425 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-02-20T08:00:21.490 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-02-20T08:00:21.566 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-02-20T08:00:21.652 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-02-20T08:00:21.716 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-02-20T08:00:21.772 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-02-20T08:00:21.868 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-02-20T08:00:21.902 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-02-20T08:00:21.999 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-02-20T08:00:22.065 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-02-20T08:00:22.161 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-02-20T08:00:22.290 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-02-20T08:00:22.442 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-02-20T08:00:22.624 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2021-02-20T08:00:22.882 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:00:22 +0000 (0:00:22.713) 0:00:31.186 ***** 2021-02-20T08:00:22.948 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2021-02-20T08:00:22.990 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:00:22 +0000 (0:00:00.107) 0:00:31.293 ***** 2021-02-20T08:00:23.068 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ changed: [smithi090.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2021-02-20T08:00:23.178 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2021-02-20T08:00:23.181 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora 2021-02-20T08:00:23.183 INFO:teuthology.task.ansible.out:rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265 2021-02-20T08:00:23.185 INFO:teuthology.task.ansible.out:c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancic 2021-02-20T08:00:23.186 INFO:teuthology.task.ansible.out:h", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1Wi 2021-02-20T08:00:23.188 INFO:teuthology.task.ansible.out:RenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vum 2021-02-20T08:00:23.189 INFO:teuthology.task.ansible.out:rao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69 2021-02-20T08:00:23.191 INFO:teuthology.task.ansible.out:mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d8 2021-02-20T08:00:23.193 INFO:teuthology.task.ansible.out:41a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix" 2021-02-20T08:00:23.194 INFO:teuthology.task.ansible.out:, "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffb 2021-02-20T08:00:23.196 INFO:teuthology.task.ansible.out:abdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec 2021-02-20T08:00:23.197 INFO:teuthology.task.ansible.out:8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f003 2021-02-20T08:00:23.199 INFO:teuthology.task.ansible.out:9212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a7190 2021-02-20T08:00:23.200 INFO:teuthology.task.ansible.out:8d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e8 2021-02-20T08:00:23.202 INFO:teuthology.task.ansible.out:84fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}]}, "changed": false} 2021-02-20T08:00:23.250 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:00:23 +0000 (0:00:00.260) 0:00:31.554 ***** 2021-02-20T08:00:23.302 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:00:23 +0000 (0:00:00.052) 0:00:31.606 ***** 2021-02-20T08:00:23.335 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2021-02-20T08:00:23.391 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:00:23 +0000 (0:00:00.088) 0:00:31.695 ***** 2021-02-20T08:00:23.454 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2021-02-20T08:00:23.455 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora 2021-02-20T08:00:23.457 INFO:teuthology.task.ansible.out:rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265 2021-02-20T08:00:23.459 INFO:teuthology.task.ansible.out:c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancic 2021-02-20T08:00:23.461 INFO:teuthology.task.ansible.out:h", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1Wi 2021-02-20T08:00:23.465 INFO:teuthology.task.ansible.out:RenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vum 2021-02-20T08:00:23.468 INFO:teuthology.task.ansible.out:rao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69 2021-02-20T08:00:23.478 INFO:teuthology.task.ansible.out:mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 2021-02-20T08:00:23.479 INFO:teuthology.task.ansible.out:5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d8 2021-02-20T08:00:23.481 INFO:teuthology.task.ansible.out:41a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveirada 2021-02-20T08:00:23.483 INFO:teuthology.task.ansible.out:n@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOe 2021-02-20T08:00:23.491 INFO:teuthology.task.ansible.out:nVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9 2021-02-20T08:00:23.493 INFO:teuthology.task.ansible.out:gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": 2021-02-20T08:00:23.494 INFO:teuthology.task.ansible.out:"dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NI 2021-02-20T08:00:23.496 INFO:teuthology.task.ansible.out:Nj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395 2021-02-20T08:00:23.496 INFO:teuthology.task.ansible.out:862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}]}, "changed": false} 2021-02-20T08:00:23.561 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:00:23 +0000 (0:00:00.169) 0:00:31.864 ***** 2021-02-20T08:00:25.116 INFO:teuthology.task.ansible.out:FAILED - RETRYING: Clone the keys repo (5 retries left). 2021-02-20T08:00:25.150 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2021-02-20T08:00:25.152 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com -> localhost] => {"after": "df8946f08e870e2d2565b79099289ca48bd825f3", "attempts": 1, "before": "df8946f08e870e2d2565b79099289ca48bd825f3", "changed": false, "remote_url_changed": false} 2021-02-20T08:00:25.199 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:00:25 +0000 (0:00:01.638) 0:00:33.503 ***** 2021-02-20T08:00:25.425 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-02-20T08:00:25.588 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-02-20T08:00:25.740 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-02-20T08:00:25.905 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZ 2021-02-20T08:00:25.907 INFO:teuthology.task.ansible.out:tIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-02-20T08:00:26.055 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-02-20T08:00:26.252 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2021-02-20T08:00:26.254 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-02-20T08:00:26.402 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-02-20T08:00:26.561 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-02-20T08:00:26.722 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-02-20T08:00:26.735 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:26.886 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2021-02-20T08:00:26.888 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-02-20T08:00:27.047 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemer 2021-02-20T08:00:27.049 INFO:teuthology.task.ansible.out:son", "validate_certs": true} skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:27.058 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:27.198 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-02-20T08:00:27.366 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2021-02-20T08:00:27.369 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-02-20T08:00:27.502 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-02-20T08:00:27.661 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2021-02-20T08:00:27.801 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-02-20T08:00:27.952 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-02-20T08:00:28.105 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-02-20T08:00:28.257 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2021-02-20T08:00:28.259 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-02-20T08:00:28.424 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2021-02-20T08:00:28.426 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-02-20T08:00:28.565 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-02-20T08:00:28.702 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-02-20T08:00:28.863 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46 2021-02-20T08:00:28.865 INFO:teuthology.task.ansible.out:TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-02-20T08:00:29.003 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-02-20T08:00:29.197 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-02-20T08:00:29.371 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-02-20T08:00:29.380 INFO:teuthology.task.ansible.out: 2021-02-20T08:00:29.551 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-02-20T08:00:29.708 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-02-20T08:00:29.872 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2021-02-20T08:00:29.874 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-02-20T08:00:30.027 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-02-20T08:00:30.179 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-02-20T08:00:30.340 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-02-20T08:00:30.513 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-02-20T08:00:30.666 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2021-02-20T08:00:30.807 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-02-20T08:00:30.959 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-02-20T08:00:31.145 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-02-20T08:00:31.317 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-02-20T08:00:31.484 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2021-02-20T08:00:31.486 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-02-20T08:00:31.704 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2021-02-20T08:00:31.706 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-02-20T08:00:31.861 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-02-20T08:00:31.863 INFO:teuthology.task.ansible.out: 2021-02-20T08:00:32.030 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-02-20T08:00:32.219 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-02-20T08:00:32.378 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2021-02-20T08:00:32.381 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-02-20T08:00:32.602 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2021-02-20T08:00:32.604 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2021-02-20T08:00:32.606 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-02-20T08:00:32.779 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-02-20T08:00:32.941 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-02-20T08:00:33.105 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-02-20T08:00:33.257 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-02-20T08:00:33.473 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-02-20T08:00:33.649 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home 2021-02-20T08:00:33.651 INFO:teuthology.task.ansible.out:/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-02-20T08:00:33.844 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-02-20T08:00:34.017 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-02-20T08:00:34.189 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2021-02-20T08:00:34.191 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} 2021-02-20T08:00:34.341 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-02-20T08:00:34.507 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-02-20T08:00:34.729 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2021-02-20T08:00:34.730 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-02-20T08:00:34.975 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-02-20T08:00:35.135 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-02-20T08:00:35.354 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-02-20T08:00:35.623 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-02-20T08:00:35.767 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-02-20T08:00:35.905 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-02-20T08:00:36.015 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* ok: [smithi090.front.sepia.ceph.com -> localhost] => {"after": "df8946f08e870e2d2565b79099289ca48bd825f3", "attempts": 2, "before": "df8946f08e870e2d2565b79099289ca48bd825f3", "changed": false, "remote_url_changed": false} 2021-02-20T08:00:36.053 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:00:36 +0000 (0:00:10.853) 0:00:44.357 ***** 2021-02-20T08:00:36.090 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:36.092 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-02-20T08:00:36.213 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-02-20T08:00:36.348 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-02-20T08:00:36.402 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-02-20T08:00:36.481 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-02-20T08:00:36.483 INFO:teuthology.task.ansible.out: 2021-02-20T08:00:36.549 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-02-20T08:00:36.628 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-02-20T08:00:36.630 INFO:teuthology.task.ansible.out: 2021-02-20T08:00:36.694 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-02-20T08:00:36.804 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZ 2021-02-20T08:00:36.806 INFO:teuthology.task.ansible.out:tIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-02-20T08:00:36.871 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-02-20T08:00:36.987 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-02-20T08:00:37.029 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-02-20T08:00:37.212 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2021-02-20T08:00:37.214 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-02-20T08:00:37.404 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-02-20T08:00:37.415 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-02-20T08:00:37.604 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-02-20T08:00:37.619 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-02-20T08:00:37.761 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-02-20T08:00:37.772 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-02-20T08:00:37.784 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:37.926 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2021-02-20T08:00:37.928 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-02-20T08:00:37.938 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-02-20T08:00:37.940 INFO:teuthology.task.ansible.out: 2021-02-20T08:00:38.081 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemer 2021-02-20T08:00:38.083 INFO:teuthology.task.ansible.out:son", "validate_certs": true} 2021-02-20T08:00:38.093 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:38.115 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:38.127 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-02-20T08:00:38.267 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-02-20T08:00:38.270 INFO:teuthology.task.ansible.out: ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-02-20T08:00:38.390 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2021-02-20T08:00:38.393 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-02-20T08:00:38.403 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1 2021-02-20T08:00:38.405 INFO:teuthology.task.ansible.out:lXTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-02-20T08:00:38.600 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-02-20T08:00:38.612 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-02-20T08:00:38.817 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2021-02-20T08:00:38.828 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-02-20T08:00:39.052 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-02-20T08:00:39.124 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-02-20T08:00:39.243 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-02-20T08:00:39.320 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-02-20T08:00:39.411 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-02-20T08:00:39.472 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.locald 2021-02-20T08:00:39.474 INFO:teuthology.task.ansible.out:omain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-02-20T08:00:39.546 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2021-02-20T08:00:39.548 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-02-20T08:00:39.651 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2021-02-20T08:00:39.653 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-02-20T08:00:39.738 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2021-02-20T08:00:39.739 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-02-20T08:00:39.830 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-02-20T08:00:39.885 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-02-20T08:00:40.016 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-02-20T08:00:40.039 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-02-20T08:00:40.194 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-02-20T08:00:40.216 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46 2021-02-20T08:00:40.218 INFO:teuthology.task.ansible.out:TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-02-20T08:00:40.367 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-02-20T08:00:40.378 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-02-20T08:00:40.515 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-02-20T08:00:40.527 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-02-20T08:00:40.672 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-02-20T08:00:40.694 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-02-20T08:00:40.846 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-02-20T08:00:40.880 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-02-20T08:00:41.016 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-02-20T08:00:41.049 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loc 2021-02-20T08:00:41.051 INFO:teuthology.task.ansible.out:al", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-02-20T08:00:41.179 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2021-02-20T08:00:41.181 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-02-20T08:00:41.191 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-02-20T08:00:41.310 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-02-20T08:00:41.343 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-02-20T08:00:41.514 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-02-20T08:00:41.516 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-02-20T08:00:41.675 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-02-20T08:00:41.688 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-02-20T08:00:41.837 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-02-20T08:00:41.850 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-02-20T08:00:42.021 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-02-20T08:00:42.024 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2021-02-20T08:00:42.208 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2021-02-20T08:00:42.211 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-02-20T08:00:42.393 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-02-20T08:00:42.395 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-02-20T08:00:42.566 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-02-20T08:00:42.599 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-02-20T08:00:42.710 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-02-20T08:00:42.749 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-02-20T08:00:42.881 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-02-20T08:00:42.935 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2021-02-20T08:00:42.937 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-02-20T08:00:43.087 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-02-20T08:00:43.120 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2021-02-20T08:00:43.121 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-02-20T08:00:43.261 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-02-20T08:00:43.284 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-02-20T08:00:43.394 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-02-20T08:00:43.501 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-02-20T08:00:43.582 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-02-20T08:00:43.688 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-02-20T08:00:43.742 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2021-02-20T08:00:43.744 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-02-20T08:00:43.851 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-02-20T08:00:43.890 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2021-02-20T08:00:43.898 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2021-02-20T08:00:43.901 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-02-20T08:00:44.021 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-02-20T08:00:44.079 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-02-20T08:00:44.233 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-02-20T08:00:44.245 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-02-20T08:00:44.423 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-02-20T08:00:44.424 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-02-20T08:00:44.569 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-02-20T08:00:44.573 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-02-20T08:00:44.714 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-02-20T08:00:44.746 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-02-20T08:00:44.876 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home 2021-02-20T08:00:44.878 INFO:teuthology.task.ansible.out:/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-02-20T08:00:45.017 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-02-20T08:00:45.056 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-02-20T08:00:45.175 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-02-20T08:00:45.219 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-02-20T08:00:45.385 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-02-20T08:00:45.387 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2021-02-20T08:00:45.393 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} 2021-02-20T08:00:45.538 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-02-20T08:00:45.563 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjL 2021-02-20T08:00:45.565 INFO:teuthology.task.ansible.out:d+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-02-20T08:00:45.733 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-02-20T08:00:45.746 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-02-20T08:00:45.880 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2021-02-20T08:00:45.882 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-02-20T08:00:45.914 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-02-20T08:00:46.066 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-02-20T08:00:46.078 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-02-20T08:00:46.239 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-02-20T08:00:46.262 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-02-20T08:00:46.412 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-02-20T08:00:46.477 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-02-20T08:00:46.615 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-02-20T08:00:46.660 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-02-20T08:00:46.757 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-02-20T08:00:46.812 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-02-20T08:00:46.930 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-02-20T08:00:46.973 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-02-20T08:00:47.085 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-02-20T08:00:47.150 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-02-20T08:00:47.260 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-02-20T08:00:47.369 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-02-20T08:00:47.455 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-02-20T08:00:47.531 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-02-20T08:00:47.598 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-02-20T08:00:47.710 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-02-20T08:00:47.784 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-02-20T08:00:47.887 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-02-20T08:00:48.000 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-02-20T08:00:48.092 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-02-20T08:00:48.141 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-02-20T08:00:48.261 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-02-20T08:00:48.321 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-02-20T08:00:48.441 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-02-20T08:00:48.511 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com", "key_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-02-20T08:00:48.586 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-02-20T08:00:48.702 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-02-20T08:00:48.755 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-02-20T08:00:48.808 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:00:48 +0000 (0:00:12.750) 0:00:57.107 ***** 2021-02-20T08:00:48.867 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-02-20T08:00:48.869 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:48.878 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:48.889 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:48.891 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-02-20T08:00:48.911 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:48.933 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:48.944 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:48.945 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:48.956 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:48.967 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:48.979 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.061 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:49.063 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-02-20T08:00:49.158 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-02-20T08:00:49.160 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-02-20T08:00:49.182 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.203 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.205 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-02-20T08:00:49.345 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-02-20T08:00:49.347 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-02-20T08:00:49.383 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:49.386 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-02-20T08:00:49.522 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-02-20T08:00:49.527 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-02-20T08:00:49.535 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTl 2021-02-20T08:00:49.537 INFO:teuthology.task.ansible.out:oAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1lXTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-02-20T08:00:49.551 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.558 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.592 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.612 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.624 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.635 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.647 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.648 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.660 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.681 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.693 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:49.694 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, 2021-02-20T08:00:49.696 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.705 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.738 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.747 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.770 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.782 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.804 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.817 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.819 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.829 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.842 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.854 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.879 INFO:teuthology.task.ansible.out: 2021-02-20T08:00:49.882 INFO:teuthology.task.ansible.out:TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:49.884 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-02-20T08:00:49.885 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.896 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.918 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.930 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.943 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.979 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.982 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:49.996 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.009 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.020 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.055 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5 2021-02-20T08:00:50.057 INFO:teuthology.task.ansible.out:b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.066 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.078 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.089 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.100 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.111 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.125 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.150 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.158 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.172 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.173 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.184 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.196 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.208 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.219 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.241 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.244 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.255 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.270 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.277 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.288 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.300 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.311 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.323 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.325 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.335 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.350 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.359 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.370 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc 2021-02-20T08:00:50.372 INFO:teuthology.task.ansible.out:+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.localdomain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.382 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.384 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.393 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.405 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.416 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.427 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.436 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.438 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.450 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.462 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.472 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.484 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.495 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.506 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.520 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.528 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.530 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08R 2021-02-20T08:00:50.532 INFO:teuthology.task.ansible.out:rDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.541 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.552 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.579 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.587 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.589 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.599 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.606 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.623 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.627 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.635 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.639 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.649 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.661 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.673 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.685 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.696 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.698 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.707 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:50.709 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073 2021-02-20T08:00:50.711 INFO:teuthology.task.ansible.out:aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.730 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.741 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.773 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.784 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.796 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.807 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.818 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.830 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.841 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.843 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.853 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.865 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.866 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-02-20T08:00:50.877 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.888 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:50.964 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:00:50 +0000 (0:00:02.160) 0:00:59.268 ***** 2021-02-20T08:00:51.027 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2021-02-20T08:00:51.028 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:51.065 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:00:51 +0000 (0:00:00.101) 0:00:59.369 ***** 2021-02-20T08:00:51.074 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:51.077 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-02-20T08:00:51.154 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-02-20T08:00:51.242 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:51.244 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-02-20T08:00:51.307 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:51.308 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-02-20T08:00:51.363 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:51.365 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-02-20T08:00:51.440 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:51.441 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-02-20T08:00:51.505 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:51.507 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-02-20T08:00:51.573 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:51.574 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-02-20T08:00:51.645 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi003.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-02-20T08:00:51.747 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q 2021-02-20T08:00:51.757 INFO:teuthology.task.ansible.out:+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.local", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-02-20T08:00:51.787 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:51.788 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-02-20T08:00:51.917 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:51.919 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} TASK [users : Remove revoked users] ******************************************** ok: [smithi003.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-02-20T08:00:52.062 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} TASK [users : Remove revoked users] ******************************************** ok: [smithi003.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-02-20T08:00:52.200 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-02-20T08:00:52.211 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:52.213 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-02-20T08:00:52.361 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-02-20T08:00:52.363 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi003.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-02-20T08:00:52.479 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-02-20T08:00:52.491 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:52.493 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-02-20T08:00:52.609 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:52.611 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-02-20T08:00:52.631 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-02-20T08:00:52.737 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:52.739 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-02-20T08:00:52.759 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:52.761 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2021-02-20T08:00:52.763 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-02-20T08:00:52.877 INFO:teuthology.task.ansible.out: 2021-02-20T08:00:52.879 INFO:teuthology.task.ansible.out:TASK [users : Remove revoked users] ******************************************** ok: [smithi003.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-02-20T08:00:52.890 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:52.892 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-02-20T08:00:52.998 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:53.000 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-02-20T08:00:53.010 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-02-20T08:00:53.137 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:53.139 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-02-20T08:00:53.160 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:53.162 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-02-20T08:00:53.267 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:53.269 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-02-20T08:00:53.310 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:53.312 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-02-20T08:00:53.393 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi003.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-02-20T08:00:53.454 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-02-20T08:00:53.549 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:53.551 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-02-20T08:00:53.604 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-02-20T08:00:53.690 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:53.692 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-02-20T08:00:53.743 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:53.745 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-02-20T08:00:53.808 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:53.810 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-02-20T08:00:53.872 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:53.874 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-02-20T08:00:53.958 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:53.960 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-02-20T08:00:54.015 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-02-20T08:00:54.102 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:54.103 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2021-02-20T08:00:54.106 INFO:teuthology.task.ansible.out: 2021-02-20T08:00:54.160 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:54.162 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-02-20T08:00:54.236 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:54.238 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-02-20T08:00:54.302 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:54.304 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-02-20T08:00:54.366 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:54.368 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-02-20T08:00:54.425 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-02-20T08:00:54.521 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:54.522 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-02-20T08:00:54.576 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:54.578 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-02-20T08:00:54.651 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:54.654 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-02-20T08:00:54.726 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-02-20T08:00:54.800 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi003.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-02-20T08:00:54.874 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:54.876 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-02-20T08:00:54.951 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:54.952 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-02-20T08:00:55.036 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:55.039 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-02-20T08:00:55.111 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:55.113 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-02-20T08:00:55.255 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:55.257 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-02-20T08:00:55.288 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:55.290 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-02-20T08:00:55.377 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:55.379 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-02-20T08:00:55.477 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:55.479 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-02-20T08:00:55.563 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:55.564 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjL 2021-02-20T08:00:55.566 INFO:teuthology.task.ansible.out:d+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-02-20T08:00:55.639 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi003.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-02-20T08:00:55.720 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-02-20T08:00:55.794 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:55.796 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-02-20T08:00:55.873 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:55.875 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-02-20T08:00:55.928 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:55.929 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-02-20T08:00:56.026 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:56.028 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-02-20T08:00:56.081 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:56.083 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-02-20T08:00:56.198 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:56.201 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-02-20T08:00:56.221 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:56.222 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-02-20T08:00:56.317 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:56.319 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-02-20T08:00:56.371 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:56.373 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-02-20T08:00:56.511 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:56.703 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} TASK [users : Remove revoked users] ******************************************** ok: [smithi003.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-02-20T08:00:56.712 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:56.715 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-02-20T08:00:56.880 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-02-20T08:00:57.012 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-02-20T08:00:57.076 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** changed: [smithi003.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "zyan", "name": "zyan", "remove": false, "state": "absent"} 2021-02-20T08:00:57.134 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:57.136 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-02-20T08:00:57.242 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi003.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-02-20T08:00:57.309 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-02-20T08:00:57.383 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi003.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-02-20T08:00:57.472 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:57.474 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-02-20T08:00:57.518 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:57.520 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-02-20T08:00:57.615 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:57.618 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-02-20T08:00:57.679 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:57.681 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-02-20T08:00:57.770 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:57.772 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-02-20T08:00:57.816 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:57.818 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-02-20T08:00:57.907 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-02-20T08:00:57.951 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:57.954 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-02-20T08:00:58.104 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:58.106 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-02-20T08:00:58.136 INFO:teuthology.task.ansible.out: 2021-02-20T08:00:58.137 INFO:teuthology.task.ansible.out:TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:58.144 INFO:teuthology.task.ansible.out: ok: [smithi003.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-02-20T08:00:58.319 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-02-20T08:00:58.320 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:58.321 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com", "key_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-02-20T08:00:58.511 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-02-20T08:00:58.513 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi003.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-02-20T08:00:58.541 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T08:00:58.643 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:00:58 +0000 (0:00:07.577) 0:01:06.947 ***** 2021-02-20T08:00:58.671 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:00:58.673 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-02-20T08:00:58.754 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:00:58 +0000 (0:00:00.111) 0:01:07.058 ***** 2021-02-20T08:00:58.780 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-02-20T08:00:58.782 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:58.783 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ 2021-02-20T08:00:58.785 INFO:teuthology.task.ansible.out:ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:58.785 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcN 2021-02-20T08:00:58.787 INFO:teuthology.task.ansible.out:E/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:58.806 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:58.809 INFO:teuthology.task.ansible.out: 2021-02-20T08:00:58.822 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:58.837 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.050 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-02-20T08:00:59.071 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.084 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.245 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-02-20T08:00:59.300 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-02-20T08:00:59.302 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => {"changed": true, "checksum": "5ebc9fcf8859a9c716ab10089a7bec7e44d29368", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "ee06ed012f51c2e3764171d4b9180349", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613808058.8009548-229308726666393/source", "state": "file", "uid": 0} 2021-02-20T08:00:59.303 INFO:teuthology.task.ansible.out: 2021-02-20T08:00:59.382 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:00:59 +0000 (0:00:00.627) 0:01:07.685 ***** 2021-02-20T08:00:59.414 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** ok: [smithi090.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-02-20T08:00:59.425 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.447 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-02-20T08:00:59.451 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => { "msg": "Host smithi003.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.517 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:00:59 +0000 (0:00:00.135) 0:01:07.821 ***** 2021-02-20T08:00:59.546 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.551 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.554 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.556 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.558 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.558 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.572 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.573 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-02-20T08:00:59.574 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-02-20T08:00:59.576 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.666 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:00:59 +0000 (0:00:00.148) 0:01:07.970 ***** 2021-02-20T08:00:59.674 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.676 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.676 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxiche 2021-02-20T08:00:59.678 INFO:teuthology.task.ansible.out:n", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.689 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.700 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-02-20T08:00:59.702 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.756 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:00:59 +0000 (0:00:00.088) 0:01:08.058 ***** 2021-02-20T08:00:59.767 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.769 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.771 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.772 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.772 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.773 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.795 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.806 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.809 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.871 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:00:59 +0000 (0:00:00.116) 0:01:08.174 ***** 2021-02-20T08:00:59.882 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.885 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.899 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.912 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.914 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:00:59.968 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:00:59 +0000 (0:00:00.097) 0:01:08.271 ***** 2021-02-20T08:01:00.000 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.002 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.003 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d16 2021-02-20T08:01:00.005 INFO:teuthology.task.ansible.out:7c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.015 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.027 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.028 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.151 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:00 +0000 (0:00:00.182) 0:01:08.454 ***** 2021-02-20T08:01:00.196 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.211 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.214 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacro 2021-02-20T08:01:00.245 INFO:teuthology.task.ansible.out:ix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023 2021-02-20T08:01:00.247 INFO:teuthology.task.ansible.out:c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional resu 2021-02-20T08:01:00.253 INFO:teuthology.task.ansible.out:lt was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.256 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.258 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.326 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:00 +0000 (0:00:00.175) 0:01:08.630 ***** 2021-02-20T08:01:00.347 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7d 2021-02-20T08:01:00.367 INFO:teuthology.task.ansible.out:ded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn 2021-02-20T08:01:00.392 INFO:teuthology.task.ansible.out:": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.393 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.394 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.402 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.502 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:00 +0000 (0:00:00.174) 0:01:08.805 ***** 2021-02-20T08:01:00.515 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.534 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.546 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.548 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.606 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:00 +0000 (0:00:00.104) 0:01:08.910 ***** 2021-02-20T08:01:00.626 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-02-20T08:01:00.627 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.629 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.631 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.642 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.659 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.673 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.682 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.693 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.695 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.733 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:00 +0000 (0:00:00.126) 0:01:09.036 ***** 2021-02-20T08:01:00.749 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.779 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.782 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.782 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.818 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.829 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.840 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-02-20T08:01:00.843 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi003.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T08:01:00.907 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:00 +0000 (0:00:00.174) 0:01:09.210 ***** 2021-02-20T08:01:00.923 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-02-20T08:01:00.926 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": 2021-02-20T08:01:00.929 INFO:teuthology.task.ansible.out: "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.939 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.950 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:00.991 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:01.025 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:01 +0000 (0:00:00.117) 0:01:09.328 ***** 2021-02-20T08:01:01.035 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:01.039 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:01.061 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:01.071 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:01.077 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-02-20T08:01:01.080 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:01.149 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:01 +0000 (0:00:00.124) 0:01:09.453 ***** 2021-02-20T08:01:01.161 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-02-20T08:01:01.163 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:01.164 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:01.170 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:01.208 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:01.212 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:01.214 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:01.226 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-02-20T08:01:01.227 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T08:01:01.229 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-02-20T08:01:01.232 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:01.236 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:01.333 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:01 +0000 (0:00:00.183) 0:01:09.636 ***** 2021-02-20T08:01:01.369 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:01.377 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:01.427 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-02-20T08:01:01.429 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:01.478 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:01 +0000 (0:00:00.145) 0:01:09.781 ***** 2021-02-20T08:01:01.572 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:01 +0000 (0:00:00.094) 0:01:09.876 ***** 2021-02-20T08:01:01.581 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-02-20T08:01:01.584 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T08:01:01.625 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:01 +0000 (0:00:00.052) 0:01:09.928 ***** 2021-02-20T08:01:01.640 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:01.701 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:01 +0000 (0:00:00.076) 0:01:10.005 ***** 2021-02-20T08:01:01.708 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-02-20T08:01:01.711 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:01.762 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:01 +0000 (0:00:00.061) 0:01:10.066 ***** 2021-02-20T08:01:01.804 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-02-20T08:01:01.807 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:01.841 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:01 +0000 (0:00:00.078) 0:01:10.145 ***** 2021-02-20T08:01:01.871 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-02-20T08:01:01.873 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:01.976 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:01 +0000 (0:00:00.134) 0:01:10.280 ***** 2021-02-20T08:01:01.988 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:01:01.991 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-02-20T08:01:02.024 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:02.065 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:02 +0000 (0:00:00.088) 0:01:10.368 ***** 2021-02-20T08:01:02.071 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:01:02.073 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-02-20T08:01:02.098 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi003.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T08:01:02.137 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:02 +0000 (0:00:00.071) 0:01:10.440 ***** 2021-02-20T08:01:02.169 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-02-20T08:01:02.170 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:02.214 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:02 +0000 (0:00:00.077) 0:01:10.518 ***** 2021-02-20T08:01:02.232 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:01:02.235 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-02-20T08:01:02.244 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:02.286 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:02 +0000 (0:00:00.071) 0:01:10.589 ***** 2021-02-20T08:01:02.351 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:01:02.354 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-02-20T08:01:02.354 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:02.418 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:02 +0000 (0:00:00.132) 0:01:10.722 ***** 2021-02-20T08:01:02.462 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:02.512 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:02 +0000 (0:00:00.094) 0:01:10.816 ***** 2021-02-20T08:01:02.532 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:01:02.539 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-02-20T08:01:02.567 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-02-20T08:01:02.570 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:02.622 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:02 +0000 (0:00:00.109) 0:01:10.926 ***** 2021-02-20T08:01:02.662 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:01:02.668 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-02-20T08:01:02.669 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:02.716 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:02 +0000 (0:00:00.091) 0:01:11.017 ***** 2021-02-20T08:01:02.777 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-02-20T08:01:02.780 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T08:01:02.851 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:02 +0000 (0:00:00.137) 0:01:11.155 ***** 2021-02-20T08:01:02.863 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:01:02.865 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-02-20T08:01:02.898 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:02.949 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:02 +0000 (0:00:00.098) 0:01:11.253 ***** 2021-02-20T08:01:02.962 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:01:02.970 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-02-20T08:01:02.984 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-02-20T08:01:02.990 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:03.045 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:03 +0000 (0:00:00.095) 0:01:11.349 ***** 2021-02-20T08:01:03.074 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:03.076 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi090.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-02-20T08:01:03.119 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:03 +0000 (0:00:00.074) 0:01:11.423 ***** 2021-02-20T08:01:03.157 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-02-20T08:01:03.158 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:03.160 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:03.232 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:03 +0000 (0:00:00.107) 0:01:11.530 ***** 2021-02-20T08:01:03.248 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:01:03.250 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-02-20T08:01:03.282 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-02-20T08:01:03.284 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-02-20T08:01:03.325 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:03 +0000 (0:00:00.098) 0:01:11.629 ***** 2021-02-20T08:01:03.364 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:01:03.367 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} TASK [common : Clean yum cache] ************************************************ skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:03.413 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:03 +0000 (0:00:00.087) 0:01:11.716 ***** 2021-02-20T08:01:03.515 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:01:03.517 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-02-20T08:01:03.591 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-02-20T08:01:03.593 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.016926", "end": "2021-02-20 08:01:03.649221", "rc": 0, "start": "2021-02-20 08:01:03.632295", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-02-20T08:01:03.638 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:03 +0000 (0:00:00.225) 0:01:11.941 ***** 2021-02-20T08:01:03.646 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:01:03.648 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-02-20T08:01:03.681 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2021-02-20T08:01:03.683 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:03.718 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:03 +0000 (0:00:00.080) 0:01:12.022 ***** 2021-02-20T08:01:03.768 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-02-20T08:01:03.769 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:03.803 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:03 +0000 (0:00:00.085) 0:01:12.107 ***** 2021-02-20T08:01:03.812 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:01:03.813 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-02-20T08:01:03.856 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-02-20T08:01:03.858 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "skip_reason": "Conditional result was False"} 2021-02-20T08:01:03.867 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "skip_reason": "Conditional result was False"} 2021-02-20T08:01:03.906 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "skip_reason": "Conditional result was False"} 2021-02-20T08:01:03.913 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "skip_reason": "Conditional result was False"} 2021-02-20T08:01:03.974 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:03 +0000 (0:00:00.170) 0:01:12.277 ***** 2021-02-20T08:01:03.985 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:01:03.986 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-02-20T08:01:03.997 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-02-20T08:01:03.999 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:04.034 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:04 +0000 (0:00:00.060) 0:01:12.338 ***** 2021-02-20T08:01:04.086 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi090.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} TASK [common : Set the timezone] *********************************************** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:04.122 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:04 +0000 (0:00:00.088) 0:01:12.426 ***** 2021-02-20T08:01:04.166 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2021-02-20T08:01:04.168 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:04.215 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:04 +0000 (0:00:00.092) 0:01:12.519 ***** 2021-02-20T08:01:04.223 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:01:04.229 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-02-20T08:01:04.253 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:04.323 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:04 +0000 (0:00:00.108) 0:01:12.627 ***** 2021-02-20T08:01:04.362 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:04.413 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:04 +0000 (0:00:00.089) 0:01:12.716 ***** 2021-02-20T08:01:04.422 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:01:04.424 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-02-20T08:01:04.489 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-02-20T08:01:04.627 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-02-20T08:01:04.751 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-02-20T08:01:04.889 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2021-02-20T08:01:05.032 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-02-20T08:01:05.149 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-02-20T08:01:05.282 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-02-20T08:01:05.419 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-02-20T08:01:05.442 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-02-20T08:01:05.444 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.885383", "end": "2021-02-20 08:01:05.500418", "rc": 0, "start": "2021-02-20 08:01:04.615035", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:01:05.480 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:05 +0000 (0:00:01.067) 0:01:13.784 ***** 2021-02-20T08:01:05.519 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-02-20T08:01:05.523 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:05.567 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:05 +0000 (0:00:00.084) 0:01:13.868 ***** 2021-02-20T08:01:05.575 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi090.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-02-20T08:01:05.699 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-02-20T08:01:05.765 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-02-20T08:01:05.767 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the apt module rather than running 'apt-get'. If you need to use command because apt is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi003.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.039133", "end": "2021-02-20 08:01:05.831634", "rc": 0, "start": "2021-02-20 08:01:05.792501", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:01:05.811 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:05 +0000 (0:00:00.246) 0:01:14.114 ***** 2021-02-20T08:01:05.830 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T08:01:05.832 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-02-20T08:01:05.943 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-02-20T08:01:06.092 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-02-20T08:01:06.245 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-02-20T08:01:06.375 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-02-20T08:01:06.496 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-02-20T08:01:06.624 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-02-20T08:01:06.766 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-02-20T08:01:06.882 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-02-20T08:01:07.019 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-02-20T08:01:07.445 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "zyan", "name": "zyan", "remove": false, "state": "absent"} 2021-02-20T08:01:07.572 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-02-20T08:01:07.701 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-02-20T08:01:07.831 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-02-20T08:01:07.951 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-02-20T08:01:08.102 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-02-20T08:01:08.232 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-02-20T08:01:08.369 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-02-20T08:01:08.518 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-02-20T08:01:08.650 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-02-20T08:01:08.809 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-02-20T08:01:08.886 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:08 +0000 (0:00:03.075) 0:01:17.190 ***** 2021-02-20T08:01:09.356 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-02-20T08:01:09.358 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"changed": true, "checksum": "5ebc9fcf8859a9c716ab10089a7bec7e44d29368", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "ee06ed012f51c2e3764171d4b9180349", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613808068.9180546-212982370499530/source", "state": "file", "uid": 0} 2021-02-20T08:01:09.427 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:09 +0000 (0:00:00.540) 0:01:17.731 ***** 2021-02-20T08:01:09.469 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-02-20T08:01:09.472 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => { "msg": "Host smithi090.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2021-02-20T08:01:09.525 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:09 +0000 (0:00:00.097) 0:01:17.828 ***** 2021-02-20T08:01:09.592 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-02-20T08:01:09.594 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-02-20T08:01:09.666 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:09 +0000 (0:00:00.141) 0:01:17.970 ***** 2021-02-20T08:01:09.699 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-02-20T08:01:09.701 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1613807738, "cache_updated": false, "changed": false} 2021-02-20T08:01:09.758 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:09 +0000 (0:00:00.091) 0:01:18.062 ***** 2021-02-20T08:01:09.766 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-02-20T08:01:09.768 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:09.809 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:09 +0000 (0:00:00.050) 0:01:18.113 ***** 2021-02-20T08:01:09.874 INFO:teuthology.task.ansible.out: 2021-02-20T08:01:09.877 INFO:teuthology.task.ansible.out:TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:09.913 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:09 +0000 (0:00:00.104) 0:01:18.217 ***** 2021-02-20T08:01:09.942 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-02-20T08:01:09.944 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:10.000 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:09 +0000 (0:00:00.086) 0:01:18.304 ***** 2021-02-20T08:01:10.032 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2021-02-20T08:01:10.034 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:10.080 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:10 +0000 (0:00:00.079) 0:01:18.383 ***** 2021-02-20T08:01:10.114 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-02-20T08:01:10.116 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:10.175 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:10 +0000 (0:00:00.094) 0:01:18.478 ***** 2021-02-20T08:01:10.203 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2021-02-20T08:01:10.205 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:10.248 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:10 +0000 (0:00:00.073) 0:01:18.552 ***** 2021-02-20T08:01:10.277 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2021-02-20T08:01:10.279 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:10.337 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:10 +0000 (0:00:00.088) 0:01:18.641 ***** 2021-02-20T08:01:10.379 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2021-02-20T08:01:10.381 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:10.418 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:10 +0000 (0:00:00.080) 0:01:18.721 ***** 2021-02-20T08:01:10.470 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi090.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi090.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T08:01:10.509 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:10 +0000 (0:00:00.091) 0:01:18.813 ***** 2021-02-20T08:01:10.537 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2021-02-20T08:01:10.539 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:10.577 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:10 +0000 (0:00:00.067) 0:01:18.881 ***** 2021-02-20T08:01:10.607 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-02-20T08:01:10.609 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:10.669 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:10 +0000 (0:00:00.091) 0:01:18.972 ***** 2021-02-20T08:01:10.698 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-02-20T08:01:10.700 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T08:01:10.747 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:10 +0000 (0:00:00.078) 0:01:19.051 ***** 2021-02-20T08:01:10.758 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** ok: [smithi003.front.sepia.ceph.com] => {"cache_update_time": 1613807738, "cache_updated": false, "changed": false} 2021-02-20T08:01:10.798 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:10 +0000 (0:00:00.051) 0:01:19.102 ***** 2021-02-20T08:01:10.805 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-02-20T08:01:10.806 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:10.847 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:10 +0000 (0:00:00.048) 0:01:19.150 ***** 2021-02-20T08:01:10.854 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-02-20T08:01:10.856 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:10.896 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:10 +0000 (0:00:00.049) 0:01:19.199 ***** 2021-02-20T08:01:10.903 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-02-20T08:01:10.905 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T08:01:10.946 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:10 +0000 (0:00:00.049) 0:01:19.249 ***** 2021-02-20T08:01:10.974 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:11.014 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:11 +0000 (0:00:00.068) 0:01:19.317 ***** 2021-02-20T08:01:11.041 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-02-20T08:01:11.043 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:11.078 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:11 +0000 (0:00:00.064) 0:01:19.382 ***** 2021-02-20T08:01:11.107 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:11.145 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:11 +0000 (0:00:00.067) 0:01:19.449 ***** 2021-02-20T08:01:11.174 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-02-20T08:01:11.178 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:11.212 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:11 +0000 (0:00:00.066) 0:01:19.516 ***** 2021-02-20T08:01:11.244 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi090.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T08:01:11.289 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:11 +0000 (0:00:00.077) 0:01:19.593 ***** 2021-02-20T08:01:11.317 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-02-20T08:01:11.319 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:11.356 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:11 +0000 (0:00:00.067) 0:01:19.660 ***** 2021-02-20T08:01:11.389 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2021-02-20T08:01:11.391 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:11.438 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:11 +0000 (0:00:00.081) 0:01:19.741 ***** 2021-02-20T08:01:11.468 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-02-20T08:01:11.470 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:11.524 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:11 +0000 (0:00:00.086) 0:01:19.828 ***** 2021-02-20T08:01:11.531 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2021-02-20T08:01:11.533 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-02-20T08:01:11.590 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:11 +0000 (0:00:00.065) 0:01:19.894 ***** 2021-02-20T08:01:11.600 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-02-20T08:01:11.603 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:11.651 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:11 +0000 (0:00:00.061) 0:01:19.955 ***** 2021-02-20T08:01:11.679 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-02-20T08:01:11.680 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:11.716 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:11 +0000 (0:00:00.064) 0:01:20.019 ***** 2021-02-20T08:01:11.754 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-02-20T08:01:11.756 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:11.794 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:11 +0000 (0:00:00.078) 0:01:20.098 ***** 2021-02-20T08:01:11.821 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-02-20T08:01:11.823 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T08:01:11.863 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:11 +0000 (0:00:00.068) 0:01:20.166 ***** 2021-02-20T08:01:11.902 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2021-02-20T08:01:11.904 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:11.945 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:11 +0000 (0:00:00.082) 0:01:20.249 ***** 2021-02-20T08:01:11.972 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-02-20T08:01:11.974 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:12.014 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:12 +0000 (0:00:00.069) 0:01:20.318 ***** 2021-02-20T08:01:12.043 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2021-02-20T08:01:12.045 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:12.081 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:12 +0000 (0:00:00.066) 0:01:20.384 ***** 2021-02-20T08:01:12.119 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-02-20T08:01:12.121 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:12.131 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:12.174 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:12 +0000 (0:00:00.092) 0:01:20.477 ***** 2021-02-20T08:01:12.211 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* skipping: [smithi090.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} 2021-02-20T08:01:12.222 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-02-20T08:01:12.259 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:12 +0000 (0:00:00.085) 0:01:20.562 ***** 2021-02-20T08:01:12.286 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-02-20T08:01:12.288 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:12.322 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:12 +0000 (0:00:00.062) 0:01:20.625 ***** 2021-02-20T08:01:12.488 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-02-20T08:01:12.490 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.015139", "end": "2021-02-20 08:01:12.561144", "rc": 0, "start": "2021-02-20 08:01:12.546005", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-02-20T08:01:12.529 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:12 +0000 (0:00:00.207) 0:01:20.833 ***** 2021-02-20T08:01:12.535 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2021-02-20T08:01:12.537 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"cache_update_time": 1613807738, "cache_updated": false, "changed": false} 2021-02-20T08:01:12.576 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:12 +0000 (0:00:00.047) 0:01:20.880 ***** 2021-02-20T08:01:12.583 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:12.623 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:12 +0000 (0:00:00.047) 0:01:20.927 ***** 2021-02-20T08:01:12.661 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-02-20T08:01:12.663 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:12.712 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:12 +0000 (0:00:00.088) 0:01:21.016 ***** 2021-02-20T08:01:12.762 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-02-20T08:01:12.763 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "skip_reason": "Conditional result was False"} 2021-02-20T08:01:12.774 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "skip_reason": "Conditional result was False"} 2021-02-20T08:01:12.787 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "skip_reason": "Conditional result was False"} 2021-02-20T08:01:12.834 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:12 +0000 (0:00:00.121) 0:01:21.137 ***** 2021-02-20T08:01:12.865 INFO:teuthology.task.ansible.out: 2021-02-20T08:01:12.867 INFO:teuthology.task.ansible.out:TASK [common : Get the current timezone] *************************************** skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:12.913 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:12 +0000 (0:00:00.079) 0:01:21.217 ***** 2021-02-20T08:01:12.934 INFO:teuthology.task.ansible.out: 2021-02-20T08:01:12.936 INFO:teuthology.task.ansible.out:TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi003.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-02-20T08:01:12.946 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2021-02-20T08:01:12.948 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:12.982 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:12 +0000 (0:00:00.068) 0:01:21.286 ***** 2021-02-20T08:01:13.019 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2021-02-20T08:01:13.021 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:13.060 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:13 +0000 (0:00:00.077) 0:01:21.363 ***** 2021-02-20T08:01:13.087 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-02-20T08:01:13.089 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:13.132 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:13 +0000 (0:00:00.072) 0:01:21.435 ***** 2021-02-20T08:01:13.160 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2021-02-20T08:01:13.162 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:13.201 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:13 +0000 (0:00:00.068) 0:01:21.504 ***** 2021-02-20T08:01:13.240 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-02-20T08:01:13.242 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-02-20T08:01:13.427 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-02-20T08:01:13.429 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.083046", "end": "2021-02-20 08:01:13.500395", "rc": 0, "start": "2021-02-20 08:01:13.417349", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:01:13.466 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:13 +0000 (0:00:00.264) 0:01:21.769 ***** 2021-02-20T08:01:13.494 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-02-20T08:01:13.496 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:13.532 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:13 +0000 (0:00:00.066) 0:01:21.836 ***** 2021-02-20T08:01:13.539 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-02-20T08:01:13.541 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-02-20T08:01:13.581 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:13 +0000 (0:00:00.048) 0:01:21.884 ***** 2021-02-20T08:01:13.711 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-02-20T08:01:13.713 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.041601", "end": "2021-02-20 08:01:13.780577", "rc": 0, "start": "2021-02-20 08:01:13.738976", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:01:13.860 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:13 +0000 (0:00:00.279) 0:01:22.163 ***** 2021-02-20T08:01:13.866 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-02-20T08:01:13.868 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T08:01:13.908 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:13 +0000 (0:00:00.048) 0:01:22.212 ***** 2021-02-20T08:01:14.234 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-02-20T08:01:14.236 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-02-20T08:01:14.521 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-02-20T08:01:14.813 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2021-02-20T08:01:14.859 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:14 +0000 (0:00:00.950) 0:01:23.163 ***** 2021-02-20T08:01:14.861 INFO:teuthology.task.ansible.out: 2021-02-20T08:01:14.900 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-02-20T08:01:14.902 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-02-20T08:01:14.938 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:14 +0000 (0:00:00.079) 0:01:23.242 ***** 2021-02-20T08:01:14.970 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2021-02-20T08:01:14.972 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:15.010 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:15 +0000 (0:00:00.071) 0:01:23.313 ***** 2021-02-20T08:01:15.038 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2021-02-20T08:01:15.040 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T08:01:15.075 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:15 +0000 (0:00:00.065) 0:01:23.379 ***** 2021-02-20T08:01:15.125 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-02-20T08:01:15.127 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-20T08:01:15.149 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-20T08:01:15.152 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-20T08:01:15.205 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:15 +0000 (0:00:00.128) 0:01:23.508 ***** 2021-02-20T08:01:15.243 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-02-20T08:01:15.245 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:15.304 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:15 +0000 (0:00:00.098) 0:01:23.606 ***** 2021-02-20T08:01:16.244 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2021-02-20T08:01:16.247 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"cache_update_time": 1613807738, "cache_updated": false, "changed": false} 2021-02-20T08:01:16.321 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:16 +0000 (0:00:01.018) 0:01:24.625 ***** 2021-02-20T08:01:16.734 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2021-02-20T08:01:16.736 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-02-20T08:01:16.820 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:16 +0000 (0:00:00.498) 0:01:25.123 ***** 2021-02-20T08:01:17.160 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-02-20T08:01:17.162 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-02-20T08:01:17.202 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:17 +0000 (0:00:00.382) 0:01:25.506 ***** 2021-02-20T08:01:17.358 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ ok: [smithi003.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-02-20T08:01:17.475 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-02-20T08:01:17.533 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:17 +0000 (0:00:00.330) 0:01:25.836 ***** 2021-02-20T08:01:17.585 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-02-20T08:01:17.587 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:17.622 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:17 +0000 (0:00:00.089) 0:01:25.926 ***** 2021-02-20T08:01:17.660 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2021-02-20T08:01:17.662 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T08:01:17.725 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:17 +0000 (0:00:00.102) 0:01:26.028 ***** 2021-02-20T08:01:17.743 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** ok: [smithi090.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1613807781, "cache_updated": false, "changed": false} 2021-02-20T08:01:17.785 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:17 +0000 (0:00:00.060) 0:01:26.089 ***** 2021-02-20T08:01:17.792 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-02-20T08:01:17.793 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:17.831 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:17 +0000 (0:00:00.046) 0:01:26.135 ***** 2021-02-20T08:01:18.259 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-02-20T08:01:18.261 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-02-20T08:01:18.304 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:18 +0000 (0:00:00.472) 0:01:26.608 ***** 2021-02-20T08:01:18.672 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-02-20T08:01:18.674 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"cache_update_time": 1613807781, "cache_updated": false, "changed": false} 2021-02-20T08:01:18.749 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:18 +0000 (0:00:00.444) 0:01:27.052 ***** 2021-02-20T08:01:18.793 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-02-20T08:01:18.796 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:18.848 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:18 +0000 (0:00:00.099) 0:01:27.152 ***** 2021-02-20T08:01:19.105 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-02-20T08:01:19.107 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-02-20 07:58:38 UTC", "ActiveEnterTimestampMonotonic": "18487684", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "nss-lookup.target basic.target network.target local-fs.target var-run.mount systemd-tmpfiles-setup.service system.slice -.mount sysinit.target remote-fs.target time-sync.target systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-02-20 07:58:38 UTC", "AssertTimestampMonotonic": "18487009", "Before": "plymouth-quit.service getty@tty1.service shutdown.target multi-user.target xdm.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-02-20 07:58:38 UTC", "ConditionTimestampMonotonic": "18487009", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-ser 2021-02-20T08:01:19.108 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1147", "ExecMainStartTimestamp": "Sat 2021-02-20 07:58:38 UTC", "ExecMainStartTimestampMonotonic": "18487655", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sat 2021-02-20 07:58:38 UTC] ; stop_time=[n/a] ; pid=1147 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-02-20 07:58:38 UTC", "InactiveExitTimestampMonotonic": "18487684", "InvocationID": "7dd1a684e19f4d0cad331cbb2501947d", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "i 2021-02-20T08:01:19.110 INFO:teuthology.task.ansible.out:nfinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128053", "LimitNPROCSoft": "128053", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128053", "LimitSIGPENDINGSoft": "128053", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1147", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2021-02-20T08:01:19.112 INFO:teuthology.task.ansible.out: "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 07:58:38 UTC", "StateChangeTimestampMonotonic": "18487684", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-02-20 07:58:38 UTC", "WatchdogTimestampMonotonic": "18487684", "WatchdogUSec": "0"}} 2021-02-20T08:01:19.151 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:19 +0000 (0:00:00.302) 0:01:27.454 ***** 2021-02-20T08:01:19.184 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-02-20T08:01:19.188 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Add krb5 config file] ******************************************* ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-02-20T08:01:19.244 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:19 +0000 (0:00:00.093) 0:01:27.548 ***** 2021-02-20T08:01:19.298 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:19 +0000 (0:00:00.053) 0:01:27.602 ***** 2021-02-20T08:01:19.320 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-02-20T08:01:19.323 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:19.376 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:19 +0000 (0:00:00.078) 0:01:27.680 ***** 2021-02-20T08:01:19.409 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2021-02-20T08:01:19.410 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:19.447 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:19 +0000 (0:00:00.071) 0:01:27.751 ***** 2021-02-20T08:01:19.487 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-02-20T08:01:19.489 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:19.529 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:19 +0000 (0:00:00.080) 0:01:27.832 ***** 2021-02-20T08:01:19.559 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-02-20T08:01:19.561 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:19.600 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:19 +0000 (0:00:00.071) 0:01:27.904 ***** 2021-02-20T08:01:19.643 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:19.680 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:19 +0000 (0:00:00.080) 0:01:27.984 ***** 2021-02-20T08:01:19.717 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-02-20T08:01:19.720 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:19.753 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:19 +0000 (0:00:00.073) 0:01:28.057 ***** 2021-02-20T08:01:19.782 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-02-20T08:01:19.783 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:19.820 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:19 +0000 (0:00:00.066) 0:01:28.124 ***** 2021-02-20T08:01:19.851 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-02-20T08:01:19.853 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:19.890 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:19 +0000 (0:00:00.069) 0:01:28.194 ***** 2021-02-20T08:01:19.924 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-02-20T08:01:19.925 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:19.962 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:19 +0000 (0:00:00.069) 0:01:28.263 ***** 2021-02-20T08:01:20.001 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-02-20T08:01:20.003 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-02-20T08:01:20.041 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:20 +0000 (0:00:00.081) 0:01:28.345 ***** 2021-02-20T08:01:20.079 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-02-20T08:01:20.081 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-02-20T08:01:20.119 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:20 +0000 (0:00:00.077) 0:01:28.422 ***** 2021-02-20T08:01:20.157 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-02-20T08:01:20.159 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2021-02-20T08:01:20.196 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:20 +0000 (0:00:00.076) 0:01:28.499 ***** 2021-02-20T08:01:20.219 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2021-02-20T08:01:20.221 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"cache_update_time": 1613807781, "cache_updated": false, "changed": false} 2021-02-20T08:01:20.256 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:20 +0000 (0:00:00.059) 0:01:28.559 ***** 2021-02-20T08:01:20.262 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-02-20T08:01:20.263 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-02-20T08:01:20.301 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:20 +0000 (0:00:00.045) 0:01:28.605 ***** 2021-02-20T08:01:20.435 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-02-20T08:01:20.437 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-02-20T08:01:20.471 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:20 +0000 (0:00:00.170) 0:01:28.775 ***** 2021-02-20T08:01:20.603 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-02-20T08:01:20.605 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} 2021-02-20T08:01:20.650 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:20 +0000 (0:00:00.177) 0:01:28.953 ***** 2021-02-20T08:01:20.656 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-02-20T08:01:20.658 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-02-20T08:01:20.690 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-02-20T08:01:20.691 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:20.751 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:20 +0000 (0:00:00.101) 0:01:29.054 ***** 2021-02-20T08:01:20.780 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-02-20T08:01:20.782 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:20.822 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:20 +0000 (0:00:00.071) 0:01:29.126 ***** 2021-02-20T08:01:20.956 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-02-20T08:01:20.958 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-02-20T08:01:21.001 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-02-20T08:01:21.003 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-02-20T08:01:21.043 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:21 +0000 (0:00:00.220) 0:01:29.346 ***** 2021-02-20T08:01:21.260 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-02-20T08:01:21.262 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-02-20T08:01:21.300 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:21 +0000 (0:00:00.257) 0:01:29.604 ***** 2021-02-20T08:01:21.424 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-02-20T08:01:21.425 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T08:01:21.461 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:21 +0000 (0:00:00.160) 0:01:29.764 ***** 2021-02-20T08:01:21.470 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-02-20T08:01:21.511 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:21 +0000 (0:00:00.050) 0:01:29.815 ***** 2021-02-20T08:01:21.550 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-02-20T08:01:21.552 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi003"}, "changed": false} 2021-02-20T08:01:21.586 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:21 +0000 (0:00:00.074) 0:01:29.889 ***** 2021-02-20T08:01:21.783 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-02-20T08:01:21.785 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-02-20T08:01:22.077 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-02-20T08:01:22.381 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2021-02-20T08:01:22.422 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:22 +0000 (0:00:00.836) 0:01:30.726 ***** 2021-02-20T08:01:22.463 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-02-20T08:01:22.464 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-02-20T08:01:22.523 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:22 +0000 (0:00:00.100) 0:01:30.827 ***** 2021-02-20T08:01:22.542 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ changed: [smithi003.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi003.front.sepia.ceph.com", "ansible_hostname": "smithi003", "ansible_nodename": "smithi003"}, "changed": true, "name": "smithi003"} 2021-02-20T08:01:22.607 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:22 +0000 (0:00:00.083) 0:01:30.911 ***** 2021-02-20T08:01:22.622 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2021-02-20T08:01:22.623 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:22.663 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:22 +0000 (0:00:00.055) 0:01:30.966 ***** 2021-02-20T08:01:22.701 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2021-02-20T08:01:22.703 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T08:01:22.737 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:22 +0000 (0:00:00.074) 0:01:31.041 ***** 2021-02-20T08:01:22.777 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-02-20T08:01:22.779 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-20T08:01:22.789 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-20T08:01:22.800 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-20T08:01:22.844 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:22 +0000 (0:00:00.106) 0:01:31.148 ***** 2021-02-20T08:01:22.876 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-02-20T08:01:22.878 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:22.932 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:22 +0000 (0:00:00.088) 0:01:31.236 ***** 2021-02-20T08:01:22.942 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2021-02-20T08:01:22.944 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-02-20T08:01:22.992 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:22 +0000 (0:00:00.059) 0:01:31.295 ***** 2021-02-20T08:01:23.311 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2021-02-20T08:01:23.313 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-02-20T08:01:23.349 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:23 +0000 (0:00:00.356) 0:01:31.652 ***** 2021-02-20T08:01:23.804 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** ok: [smithi090.front.sepia.ceph.com] => {"cache_update_time": 1613807781, "cache_updated": false, "changed": false} 2021-02-20T08:01:23.844 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:23 +0000 (0:00:00.495) 0:01:32.148 ***** 2021-02-20T08:01:23.898 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-02-20T08:01:23.900 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-02-20T08:01:23.945 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:23 +0000 (0:00:00.100) 0:01:32.248 ***** 2021-02-20T08:01:23.974 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:24.010 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:24 +0000 (0:00:00.064) 0:01:32.313 ***** 2021-02-20T08:01:24.049 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:24.101 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:24 +0000 (0:00:00.091) 0:01:32.405 ***** 2021-02-20T08:01:24.129 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-02-20T08:01:24.130 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:24.171 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:24 +0000 (0:00:00.069) 0:01:32.474 ***** 2021-02-20T08:01:24.179 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2021-02-20T08:01:24.180 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-02-20T08:01:24.224 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:24 +0000 (0:00:00.053) 0:01:32.527 ***** 2021-02-20T08:01:24.230 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-02-20T08:01:24.232 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:24.271 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:24 +0000 (0:00:00.046) 0:01:32.574 ***** 2021-02-20T08:01:24.300 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-02-20T08:01:24.302 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:24.336 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:24 +0000 (0:00:00.065) 0:01:32.640 ***** 2021-02-20T08:01:24.364 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-02-20T08:01:24.366 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:24.401 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:24 +0000 (0:00:00.064) 0:01:32.704 ***** 2021-02-20T08:01:24.440 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-02-20T08:01:24.442 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:24.478 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:24 +0000 (0:00:00.077) 0:01:32.782 ***** 2021-02-20T08:01:24.506 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-02-20T08:01:24.509 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:24.548 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:24 +0000 (0:00:00.069) 0:01:32.851 ***** 2021-02-20T08:01:24.559 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-02-20T08:01:24.560 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-02-20T08:01:24.605 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:24 +0000 (0:00:00.057) 0:01:32.909 ***** 2021-02-20T08:01:24.612 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-02-20T08:01:24.659 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:24 +0000 (0:00:00.053) 0:01:32.962 ***** 2021-02-20T08:01:24.687 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-02-20T08:01:24.727 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:24 +0000 (0:00:00.068) 0:01:33.030 ***** 2021-02-20T08:01:24.755 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-02-20T08:01:24.794 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:24 +0000 (0:00:00.067) 0:01:33.098 ***** 2021-02-20T08:01:24.802 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-02-20T08:01:24.804 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-02-20T08:01:24.825 INFO:teuthology.task.ansible.out: 2021-02-20T08:01:24.827 INFO:teuthology.task.ansible.out:TASK [testnode : Install GPG] ************************************************** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:24.871 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:24 +0000 (0:00:00.076) 0:01:33.174 ***** 2021-02-20T08:01:24.879 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-02-20T08:01:24.880 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-02-20T08:01:24.930 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:24 +0000 (0:00:00.059) 0:01:33.233 ***** 2021-02-20T08:01:24.936 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-02-20T08:01:24.938 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-02-20T08:01:24.979 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:24 +0000 (0:00:00.049) 0:01:33.283 ***** 2021-02-20T08:01:24.996 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-02-20T08:01:24.998 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:25.035 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:25 +0000 (0:00:00.055) 0:01:33.339 ***** 2021-02-20T08:01:25.041 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-02-20T08:01:25.043 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:25.082 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:25 +0000 (0:00:00.046) 0:01:33.385 ***** 2021-02-20T08:01:25.088 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2021-02-20T08:01:25.090 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T08:01:25.126 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:25 +0000 (0:00:00.045) 0:01:33.430 ***** 2021-02-20T08:01:25.133 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T08:01:25.135 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:25.175 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:25 +0000 (0:00:00.048) 0:01:33.478 ***** 2021-02-20T08:01:25.181 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-02-20T08:01:25.182 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:25.223 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:25 +0000 (0:00:00.047) 0:01:33.526 ***** 2021-02-20T08:01:25.229 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-02-20T08:01:25.231 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:25.269 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:25 +0000 (0:00:00.046) 0:01:33.573 ***** 2021-02-20T08:01:25.297 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-02-20T08:01:25.299 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:25.335 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:25 +0000 (0:00:00.065) 0:01:33.638 ***** 2021-02-20T08:01:25.364 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-02-20T08:01:25.366 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:25.401 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:25 +0000 (0:00:00.066) 0:01:33.705 ***** 2021-02-20T08:01:25.431 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-02-20T08:01:25.433 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:25.468 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:25 +0000 (0:00:00.066) 0:01:33.772 ***** 2021-02-20T08:01:25.498 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-02-20T08:01:25.500 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:25.537 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:25 +0000 (0:00:00.069) 0:01:33.841 ***** 2021-02-20T08:01:25.548 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-02-20T08:01:25.549 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-02-20T08:01:25.586 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:25 +0000 (0:00:00.048) 0:01:33.889 ***** 2021-02-20T08:01:25.594 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-02-20T08:01:25.595 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:25.639 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:25 +0000 (0:00:00.052) 0:01:33.942 ***** 2021-02-20T08:01:25.667 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-02-20T08:01:25.669 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:25.705 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:25 +0000 (0:00:00.066) 0:01:34.009 ***** 2021-02-20T08:01:25.732 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-02-20T08:01:25.734 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:25.774 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:25 +0000 (0:00:00.068) 0:01:34.078 ***** 2021-02-20T08:01:25.802 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-02-20T08:01:25.804 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:25.838 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:25 +0000 (0:00:00.064) 0:01:34.142 ***** 2021-02-20T08:01:25.846 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-02-20T08:01:25.847 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-02-20 07:59:21 UTC", "ActiveEnterTimestampMonotonic": "27905116", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target systemd-tmpfiles-setup.service systemd-journald.socket var-run.mount system.slice nss-lookup.target sysinit.target basic.target remote-fs.target -.mount local-fs.target time-sync.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-02-20 07:59:21 UTC", "AssertTimestampMonotonic": "27904139", "Before": "shutdown.target plymouth-quit.service multi-user.target getty@tty1.service xdm.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-02-20 07:59:21 UTC", "ConditionTimestampMonotonic": "27904139", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-ser 2021-02-20T08:01:25.849 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1146", "ExecMainStartTimestamp": "Sat 2021-02-20 07:59:21 UTC", "ExecMainStartTimestampMonotonic": "27905083", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sat 2021-02-20 07:59:21 UTC] ; stop_time=[n/a] ; pid=1146 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-02-20 07:59:21 UTC", "InactiveExitTimestampMonotonic": "27905116", "InvocationID": "ff1814fda0764c31806038d19fab53d4", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "i 2021-02-20T08:01:25.850 INFO:teuthology.task.ansible.out:nfinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128028", "LimitNPROCSoft": "128028", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128028", "LimitSIGPENDINGSoft": "128028", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1146", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2021-02-20T08:01:25.852 INFO:teuthology.task.ansible.out: "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 07:59:21 UTC", "StateChangeTimestampMonotonic": "27905116", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-02-20 07:59:21 UTC", "WatchdogTimestampMonotonic": "27905115", "WatchdogUSec": "0"}} 2021-02-20T08:01:25.885 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:25 +0000 (0:00:00.047) 0:01:34.189 ***** 2021-02-20T08:01:25.892 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-02-20T08:01:25.894 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:25.930 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:25 +0000 (0:00:00.043) 0:01:34.233 ***** 2021-02-20T08:01:25.937 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-02-20T08:01:25.939 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:25.976 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:25 +0000 (0:00:00.046) 0:01:34.280 ***** 2021-02-20T08:01:25.983 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-02-20T08:01:25.985 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:26.021 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:26 +0000 (0:00:00.044) 0:01:34.325 ***** 2021-02-20T08:01:26.028 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-02-20T08:01:26.029 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:26.072 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:26 +0000 (0:00:00.051) 0:01:34.376 ***** 2021-02-20T08:01:26.080 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-02-20T08:01:26.081 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:26.120 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:26 +0000 (0:00:00.047) 0:01:34.423 ***** 2021-02-20T08:01:26.129 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:26.166 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:26 +0000 (0:00:00.046) 0:01:34.470 ***** 2021-02-20T08:01:26.174 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-02-20T08:01:26.176 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:26.223 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:26 +0000 (0:00:00.056) 0:01:34.526 ***** 2021-02-20T08:01:26.230 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-02-20T08:01:26.232 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:26.275 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:26 +0000 (0:00:00.051) 0:01:34.578 ***** 2021-02-20T08:01:26.285 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2021-02-20T08:01:26.288 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:26.334 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:26 +0000 (0:00:00.058) 0:01:34.637 ***** 2021-02-20T08:01:26.340 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-02-20T08:01:26.342 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:26.386 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:26 +0000 (0:00:00.052) 0:01:34.690 ***** 2021-02-20T08:01:26.392 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-02-20T08:01:26.394 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:26.431 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:26 +0000 (0:00:00.045) 0:01:34.735 ***** 2021-02-20T08:01:26.438 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2021-02-20T08:01:26.440 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:26.475 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:26 +0000 (0:00:00.043) 0:01:34.779 ***** 2021-02-20T08:01:26.481 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-02-20T08:01:26.483 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:26.523 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:26 +0000 (0:00:00.048) 0:01:34.827 ***** 2021-02-20T08:01:26.530 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-02-20T08:01:26.532 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:26.570 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:26 +0000 (0:00:00.046) 0:01:34.873 ***** 2021-02-20T08:01:26.576 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-02-20T08:01:26.578 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:26.627 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:26 +0000 (0:00:00.057) 0:01:34.931 ***** 2021-02-20T08:01:26.641 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-02-20T08:01:26.647 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:26.690 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:26 +0000 (0:00:00.063) 0:01:34.994 ***** 2021-02-20T08:01:26.698 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-02-20T08:01:26.701 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:26.746 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:26 +0000 (0:00:00.054) 0:01:35.049 ***** 2021-02-20T08:01:26.757 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:26.806 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:26 +0000 (0:00:00.060) 0:01:35.109 ***** 2021-02-20T08:01:26.833 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-02-20T08:01:26.834 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:26.895 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:26 +0000 (0:00:00.089) 0:01:35.199 ***** 2021-02-20T08:01:26.920 INFO:teuthology.task.ansible.out: 2021-02-20T08:01:26.924 INFO:teuthology.task.ansible.out:TASK [testnode : Check for custom repos] *************************************** ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005262", "end": "2021-02-20 08:01:26.972093", "rc": 0, "start": "2021-02-20 08:01:26.966831", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:01:26.966 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:26 +0000 (0:00:00.071) 0:01:35.270 ***** 2021-02-20T08:01:26.975 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-02-20T08:01:26.977 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-02-20T08:01:27.027 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:27 +0000 (0:00:00.060) 0:01:35.330 ***** 2021-02-20T08:01:27.034 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-02-20T08:01:27.075 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:27 +0000 (0:00:00.048) 0:01:35.379 ***** 2021-02-20T08:01:27.082 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-02-20T08:01:27.084 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-02-20T08:01:27.126 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:27 +0000 (0:00:00.050) 0:01:35.429 ***** 2021-02-20T08:01:27.189 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** ok: [smithi090.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2021-02-20T08:01:27.231 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:27 +0000 (0:00:00.101) 0:01:35.531 ***** 2021-02-20T08:01:27.266 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-02-20T08:01:27.268 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-02-20T08:01:27.305 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:27 +0000 (0:00:00.077) 0:01:35.609 ***** 2021-02-20T08:01:27.457 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-02-20T08:01:27.458 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-02-20T08:01:27.508 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:27 +0000 (0:00:00.202) 0:01:35.811 ***** 2021-02-20T08:01:27.550 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-02-20T08:01:27.551 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-02-20T08:01:27.592 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:27 +0000 (0:00:00.084) 0:01:35.896 ***** 2021-02-20T08:01:27.632 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-02-20T08:01:27.634 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:27.669 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:27 +0000 (0:00:00.076) 0:01:35.973 ***** 2021-02-20T08:01:27.676 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-02-20T08:01:27.677 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} 2021-02-20T08:01:27.721 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:27 +0000 (0:00:00.052) 0:01:36.025 ***** 2021-02-20T08:01:27.750 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-02-20T08:01:27.751 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:27.788 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:27 +0000 (0:00:00.066) 0:01:36.091 ***** 2021-02-20T08:01:27.816 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-02-20T08:01:27.818 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:27.852 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:27 +0000 (0:00:00.064) 0:01:36.156 ***** 2021-02-20T08:01:28.031 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-02-20T08:01:28.032 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-02-20T08:01:28.070 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:28 +0000 (0:00:00.217) 0:01:36.373 ***** 2021-02-20T08:01:28.480 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-02-20T08:01:28.532 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:28 +0000 (0:00:00.462) 0:01:36.836 ***** 2021-02-20T08:01:28.562 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-02-20T08:01:28.564 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi090"}, "changed": false} 2021-02-20T08:01:28.602 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:28 +0000 (0:00:00.070) 0:01:36.906 ***** 2021-02-20T08:01:29.012 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2021-02-20T08:01:29.013 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi090.front.sepia.ceph.com", "ansible_hostname": "smithi090", "ansible_nodename": "smithi090"}, "changed": true, "name": "smithi090"} 2021-02-20T08:01:29.051 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:29 +0000 (0:00:00.449) 0:01:37.355 ***** 2021-02-20T08:01:29.231 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-02-20T08:01:29.233 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-02-20T08:01:29.362 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2021-02-20T08:01:29.364 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-02-20T08:01:29.400 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:29 +0000 (0:00:00.348) 0:01:37.704 ***** 2021-02-20T08:01:29.702 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2021-02-20T08:01:29.704 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-02-20T08:01:29.739 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:29 +0000 (0:00:00.338) 0:01:38.042 ***** 2021-02-20T08:01:30.177 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-02-20T08:01:30.179 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-02-20T08:01:30.216 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:30 +0000 (0:00:00.477) 0:01:38.520 ***** 2021-02-20T08:01:30.277 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-02-20T08:01:30.278 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-02-20T08:01:30.318 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:30 +0000 (0:00:00.101) 0:01:38.622 ***** 2021-02-20T08:01:30.345 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-02-20T08:01:30.347 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:30.383 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:30 +0000 (0:00:00.065) 0:01:38.687 ***** 2021-02-20T08:01:30.412 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:30.455 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:30 +0000 (0:00:00.072) 0:01:38.759 ***** 2021-02-20T08:01:30.483 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-02-20T08:01:30.485 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:30.520 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:30 +0000 (0:00:00.064) 0:01:38.824 ***** 2021-02-20T08:01:30.548 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-02-20T08:01:30.549 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:30.587 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:30 +0000 (0:00:00.066) 0:01:38.891 ***** 2021-02-20T08:01:30.614 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-02-20T08:01:30.616 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:30.649 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:30 +0000 (0:00:00.062) 0:01:38.953 ***** 2021-02-20T08:01:30.676 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-02-20T08:01:30.678 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:30.715 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:30 +0000 (0:00:00.065) 0:01:39.019 ***** 2021-02-20T08:01:30.742 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-02-20T08:01:30.744 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:30.779 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:30 +0000 (0:00:00.063) 0:01:39.082 ***** 2021-02-20T08:01:30.816 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-02-20T08:01:30.818 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:30.853 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:30 +0000 (0:00:00.074) 0:01:39.157 ***** 2021-02-20T08:01:30.880 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-02-20T08:01:30.919 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:30 +0000 (0:00:00.065) 0:01:39.222 ***** 2021-02-20T08:01:30.957 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-02-20T08:01:30.995 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:30 +0000 (0:00:00.076) 0:01:39.299 ***** 2021-02-20T08:01:31.022 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-02-20T08:01:31.060 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:31 +0000 (0:00:00.064) 0:01:39.363 ***** 2021-02-20T08:01:31.087 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2021-02-20T08:01:31.088 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:31.126 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:31 +0000 (0:00:00.066) 0:01:39.430 ***** 2021-02-20T08:01:31.145 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* ok: [smithi003.front.sepia.ceph.com] => {"cache_update_time": 1613807738, "cache_updated": false, "changed": false} 2021-02-20T08:01:31.185 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:31 +0000 (0:00:00.058) 0:01:39.489 ***** 2021-02-20T08:01:31.193 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-02-20T08:01:31.194 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-02-20T08:01:31.231 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:31 +0000 (0:00:00.045) 0:01:39.534 ***** 2021-02-20T08:01:31.240 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-02-20T08:01:31.279 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:31 +0000 (0:00:00.047) 0:01:39.582 ***** 2021-02-20T08:01:31.286 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-02-20T08:01:31.288 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:31.326 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:31 +0000 (0:00:00.047) 0:01:39.630 ***** 2021-02-20T08:01:31.354 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T08:01:31.356 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:31.392 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:31 +0000 (0:00:00.066) 0:01:39.696 ***** 2021-02-20T08:01:31.419 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-02-20T08:01:31.421 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:31.459 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:31 +0000 (0:00:00.066) 0:01:39.763 ***** 2021-02-20T08:01:31.466 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-02-20T08:01:31.468 INFO:teuthology.task.ansible.out:[WARNING]: Consider using 'become', 'become_method', and 'become_user' rather than running sudo changed: [smithi003.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.031561", "end": "2021-02-20 08:01:31.506770", "rc": 0, "start": "2021-02-20 08:01:31.475209", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:01:31.508 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:31 +0000 (0:00:00.048) 0:01:39.812 ***** 2021-02-20T08:01:31.516 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:31.553 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:31 +0000 (0:00:00.045) 0:01:39.857 ***** 2021-02-20T08:01:31.581 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-02-20T08:01:31.583 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:31.620 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:31 +0000 (0:00:00.066) 0:01:39.924 ***** 2021-02-20T08:01:31.649 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-02-20T08:01:31.650 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:31.688 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:31 +0000 (0:00:00.067) 0:01:39.991 ***** 2021-02-20T08:01:31.715 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-02-20T08:01:31.717 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:31.750 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:31 +0000 (0:00:00.062) 0:01:40.054 ***** 2021-02-20T08:01:31.790 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-02-20T08:01:31.792 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:31.839 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:31 +0000 (0:00:00.088) 0:01:40.142 ***** 2021-02-20T08:01:31.880 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-02-20T08:01:31.882 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:31.922 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:31 +0000 (0:00:00.083) 0:01:40.226 ***** 2021-02-20T08:01:31.954 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-02-20T08:01:31.955 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:31.990 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:31 +0000 (0:00:00.067) 0:01:40.293 ***** 2021-02-20T08:01:32.018 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-02-20T08:01:32.020 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:32.058 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:32 +0000 (0:00:00.067) 0:01:40.361 ***** 2021-02-20T08:01:32.085 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-02-20T08:01:32.087 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:32.122 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:32 +0000 (0:00:00.063) 0:01:40.425 ***** 2021-02-20T08:01:32.149 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-02-20T08:01:32.151 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:32.187 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:32 +0000 (0:00:00.064) 0:01:40.490 ***** 2021-02-20T08:01:32.216 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-02-20T08:01:32.218 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:32.256 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:32 +0000 (0:00:00.069) 0:01:40.560 ***** 2021-02-20T08:01:32.284 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-02-20T08:01:32.286 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:32.319 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:32 +0000 (0:00:00.063) 0:01:40.623 ***** 2021-02-20T08:01:32.350 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:32.385 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:32 +0000 (0:00:00.065) 0:01:40.689 ***** 2021-02-20T08:01:32.412 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-02-20T08:01:32.414 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:32.449 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:32 +0000 (0:00:00.064) 0:01:40.753 ***** 2021-02-20T08:01:32.489 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-02-20T08:01:32.491 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:32.524 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:32 +0000 (0:00:00.075) 0:01:40.828 ***** 2021-02-20T08:01:32.565 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-02-20T08:01:32.566 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:32.603 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:32 +0000 (0:00:00.078) 0:01:40.907 ***** 2021-02-20T08:01:32.632 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-02-20T08:01:32.634 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:32.668 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:32 +0000 (0:00:00.065) 0:01:40.972 ***** 2021-02-20T08:01:32.823 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-02-20T08:01:32.825 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.003337", "end": "2021-02-20 08:01:32.877364", "rc": 0, "start": "2021-02-20 08:01:32.874027", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:01:32.862 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:32 +0000 (0:00:00.194) 0:01:41.166 ***** 2021-02-20T08:01:32.890 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-02-20T08:01:32.930 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:32 +0000 (0:00:00.067) 0:01:41.233 ***** 2021-02-20T08:01:33.265 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-02-20T08:01:33.301 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:33 +0000 (0:00:00.371) 0:01:41.605 ***** 2021-02-20T08:01:33.327 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-02-20T08:01:33.329 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:33.365 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:33 +0000 (0:00:00.063) 0:01:41.669 ***** 2021-02-20T08:01:34.301 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-02-20T08:01:34.302 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-02-20T08:01:34.639 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2021-02-20T08:01:34.641 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"cache_update_time": 1613807738, "cache_updated": false, "changed": false} 2021-02-20T08:01:34.692 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:34 +0000 (0:00:01.326) 0:01:42.995 ***** 2021-02-20T08:01:35.237 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-02-20T08:01:35.239 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-02-20T08:01:35.277 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:35 +0000 (0:00:00.585) 0:01:43.581 ***** 2021-02-20T08:01:36.216 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-02-20T08:01:36.218 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"cache_update_time": 1613807781, "cache_updated": false, "changed": false} 2021-02-20T08:01:36.255 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:36 +0000 (0:00:00.978) 0:01:44.559 ***** 2021-02-20T08:01:36.286 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-02-20T08:01:36.329 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:36 +0000 (0:00:00.073) 0:01:44.633 ***** 2021-02-20T08:01:36.519 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-02-20T08:01:36.521 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.035530", "end": "2021-02-20 08:01:36.576402", "rc": 0, "start": "2021-02-20 08:01:36.540872", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:01:36.557 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:36 +0000 (0:00:00.227) 0:01:44.860 ***** 2021-02-20T08:01:39.407 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** ok: [smithi090.front.sepia.ceph.com] => {"cache_update_time": 1613807781, "cache_updated": false, "changed": false} 2021-02-20T08:01:39.445 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:39 +0000 (0:00:02.888) 0:01:47.749 ***** 2021-02-20T08:01:43.072 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2021-02-20T08:01:43.073 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 34 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137610 files and directorie 2021-02-20T08:01:43.075 INFO:teuthology.task.ansible.out:s currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 34 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Read 2021-02-20T08:01:43.077 INFO:teuthology.task.ansible.out:ing database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137610 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-02-20T08:01:43.111 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:43 +0000 (0:00:03.665) 0:01:51.414 ***** 2021-02-20T08:01:44.064 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-02-20T08:01:44.066 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"cache_update_time": 1613807738, "cache_updated": false, "changed": false} 2021-02-20T08:01:44.106 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:44 +0000 (0:00:00.995) 0:01:52.410 ***** 2021-02-20T08:01:44.147 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-02-20T08:01:44.149 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:44.185 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:44 +0000 (0:00:00.078) 0:01:52.488 ***** 2021-02-20T08:01:48.659 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2021-02-20T08:01:48.661 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 34 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137610 files and directorie 2021-02-20T08:01:48.663 INFO:teuthology.task.ansible.out:s currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 34 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Read 2021-02-20T08:01:48.665 INFO:teuthology.task.ansible.out:ing database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137610 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-02-20T08:01:48.708 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:48 +0000 (0:00:04.523) 0:01:57.012 ***** 2021-02-20T08:01:49.590 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-02-20T08:01:49.592 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"cache_update_time": 1613807781, "cache_updated": false, "changed": false} 2021-02-20T08:01:49.634 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:49 +0000 (0:00:00.926) 0:01:57.938 ***** 2021-02-20T08:01:49.662 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-02-20T08:01:49.664 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:49.698 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:49 +0000 (0:00:00.063) 0:01:58.002 ***** 2021-02-20T08:01:53.129 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-02-20T08:01:53.132 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => {"cache_update_time": 1613807738, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 34 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.35 [40.5 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.35 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.35 [5194 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.35 [868 kB]\nFetched 10.4 MB in 1s (9405 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database .. 2021-02-20T08:01:53.133 INFO:teuthology.task.ansible.out:. 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137468 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubunt 2021-02-20T08:01:53.135 INFO:teuthology.task.ansible.out:u1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 34 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.35 [40.5 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.35 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.35 [5194 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.35 [868 kB]", "Fetched 10.4 MB in 1s (9405 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(R 2021-02-20T08:01:53.137 INFO:teuthology.task.ansible.out:eading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137468 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-02-20T08:01:53.169 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:53 +0000 (0:00:03.470) 0:02:01.472 ***** 2021-02-20T08:01:54.110 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-02-20T08:01:54.111 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"cache_update_time": 1613807738, "cache_updated": false, "changed": false} 2021-02-20T08:01:54.147 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:54 +0000 (0:00:00.977) 0:02:02.450 ***** 2021-02-20T08:01:54.279 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-02-20T08:01:54.281 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-02-20T08:01:54.317 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:54 +0000 (0:00:00.169) 0:02:02.620 ***** 2021-02-20T08:01:54.356 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:54.391 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:54 +0000 (0:00:00.074) 0:02:02.695 ***** 2021-02-20T08:01:54.580 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-02-20T08:01:54.616 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:54 +0000 (0:00:00.224) 0:02:02.920 ***** 2021-02-20T08:01:55.150 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-02-20T08:01:55.152 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-02-20T08:01:55.192 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:55 +0000 (0:00:00.576) 0:02:03.496 ***** 2021-02-20T08:01:55.402 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-02-20T08:01:55.404 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-02-20T08:01:55.441 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:55 +0000 (0:00:00.248) 0:02:03.744 ***** 2021-02-20T08:01:55.620 INFO:teuthology.task.ansible.out: 2021-02-20T08:01:55.622 INFO:teuthology.task.ansible.out:TASK [testnode : Set up static IP in /etc/hosts.] ****************************** ok: [smithi003.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T08:01:55.670 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:55 +0000 (0:00:00.229) 0:02:03.974 ***** 2021-02-20T08:01:55.672 INFO:teuthology.task.ansible.out: 2021-02-20T08:01:56.005 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-02-20T08:01:56.006 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sat 2021-02-20 07:58:38 UTC", "ActiveEnterTimestampMonotonic": "18963891", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target system.slice nss-lookup.target basic.target -.mount remote-fs.target sysinit.target systemd-tmpfiles-setup.service systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-02-20 07:58:38 UTC", "AssertTimestampMonotonic": "18486306", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-02-20 07:58:38 UTC", "ConditionTimestampMonotonic": "18486306", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2021-02-20T08:01:56.007 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1346", "ExecMainStartTimestamp": "Sat 2021-02-20 07:58:38 UTC", "ExecMainStartTimestampMonotonic": "18963880", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-02-20 07:58:38 UTC", "InactiveExitTimestampMonotonic": "18486963", "InvocationID": "29631896adc7462d96766b5c3569ed02", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2021-02-20T08:01:56.009 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128053", "LimitNPROCSoft": "128053", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128053", "LimitSIGPENDINGSoft": "128053", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1346", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2021-02-20T08:01:56.011 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 07:58:38 UTC", "StateChangeTimestampMonotonic": "18963891", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-02-20 07:58:38 UTC", "WatchdogTimestampMonotonic": "18963890", "WatchdogUSec": "0"}} 2021-02-20T08:01:56.046 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:56 +0000 (0:00:00.376) 0:02:04.350 ***** 2021-02-20T08:01:56.073 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-02-20T08:01:56.075 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:56.112 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:56 +0000 (0:00:00.066) 0:02:04.416 ***** 2021-02-20T08:01:56.140 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-02-20T08:01:56.142 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:56.183 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:56 +0000 (0:00:00.071) 0:02:04.487 ***** 2021-02-20T08:01:56.210 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-02-20T08:01:56.212 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:56.246 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:56 +0000 (0:00:00.062) 0:02:04.550 ***** 2021-02-20T08:01:56.285 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-02-20T08:01:56.286 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:56.332 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:56 +0000 (0:00:00.085) 0:02:04.635 ***** 2021-02-20T08:01:56.360 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-02-20T08:01:56.361 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:56.401 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:56 +0000 (0:00:00.069) 0:02:04.705 ***** 2021-02-20T08:01:56.430 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-02-20T08:01:56.432 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:56.467 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:56 +0000 (0:00:00.065) 0:02:04.770 ***** 2021-02-20T08:01:56.494 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-02-20T08:01:56.495 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:56.543 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:56 +0000 (0:00:00.076) 0:02:04.847 ***** 2021-02-20T08:01:56.572 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-02-20T08:01:56.574 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:56.608 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:56 +0000 (0:00:00.065) 0:02:04.912 ***** 2021-02-20T08:01:56.636 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T08:01:56.638 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:56.675 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:56 +0000 (0:00:00.066) 0:02:04.979 ***** 2021-02-20T08:01:56.704 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-02-20T08:01:56.706 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:56.750 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:56 +0000 (0:00:00.075) 0:02:05.054 ***** 2021-02-20T08:01:56.778 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-02-20T08:01:56.779 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:56.817 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:56 +0000 (0:00:00.066) 0:02:05.120 ***** 2021-02-20T08:01:56.846 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-02-20T08:01:56.848 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:56.903 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:56 +0000 (0:00:00.086) 0:02:05.207 ***** 2021-02-20T08:01:56.932 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-02-20T08:01:56.934 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:56.983 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:56 +0000 (0:00:00.079) 0:02:05.287 ***** 2021-02-20T08:01:57.014 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-20T08:01:57.016 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:57.058 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:57 +0000 (0:00:00.074) 0:02:05.362 ***** 2021-02-20T08:01:57.087 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-02-20T08:01:57.089 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:57.124 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:57 +0000 (0:00:00.066) 0:02:05.428 ***** 2021-02-20T08:01:57.153 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-02-20T08:01:57.154 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:57.192 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:57 +0000 (0:00:00.067) 0:02:05.496 ***** 2021-02-20T08:01:57.219 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-02-20T08:01:57.221 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:57.257 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:57 +0000 (0:00:00.064) 0:02:05.560 ***** 2021-02-20T08:01:57.285 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-02-20T08:01:57.287 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:57.322 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:57 +0000 (0:00:00.065) 0:02:05.626 ***** 2021-02-20T08:01:57.357 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-02-20T08:01:57.359 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:57.396 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:57 +0000 (0:00:00.074) 0:02:05.700 ***** 2021-02-20T08:01:57.424 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-20T08:01:57.425 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:57.462 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:57 +0000 (0:00:00.065) 0:02:05.766 ***** 2021-02-20T08:01:57.489 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-02-20T08:01:57.491 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:57.525 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:57 +0000 (0:00:00.063) 0:02:05.829 ***** 2021-02-20T08:01:57.554 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-02-20T08:01:57.556 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:57.591 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:57 +0000 (0:00:00.065) 0:02:05.894 ***** 2021-02-20T08:01:57.618 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-20T08:01:57.620 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:57.655 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:57 +0000 (0:00:00.064) 0:02:05.959 ***** 2021-02-20T08:01:57.683 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:57.720 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:57 +0000 (0:00:00.064) 0:02:06.023 ***** 2021-02-20T08:01:57.747 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-02-20T08:01:57.749 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:57.785 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:57 +0000 (0:00:00.064) 0:02:06.088 ***** 2021-02-20T08:01:57.812 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2021-02-20T08:01:57.814 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:57.852 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:57 +0000 (0:00:00.067) 0:02:06.155 ***** 2021-02-20T08:01:58.002 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-02-20T08:01:58.004 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-02-20T08:01:58.041 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:58 +0000 (0:00:00.188) 0:02:06.344 ***** 2021-02-20T08:01:58.387 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-02-20T08:01:58.422 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:58 +0000 (0:00:00.381) 0:02:06.726 ***** 2021-02-20T08:01:58.450 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-02-20T08:01:58.452 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:58.493 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:58 +0000 (0:00:00.070) 0:02:06.796 ***** 2021-02-20T08:01:58.523 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-02-20T08:01:58.526 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"cache_update_time": 1613807781, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 34 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.35 [40.5 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.35 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.35 [5194 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.35 [868 kB]\nFetched 10.4 MB in 1s (9206 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database .. 2021-02-20T08:01:58.527 INFO:teuthology.task.ansible.out:. 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137468 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubunt 2021-02-20T08:01:58.529 INFO:teuthology.task.ansible.out:u1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 34 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.35 [40.5 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.35 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.35 [5194 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.35 [868 kB]", "Fetched 10.4 MB in 1s (9206 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(R 2021-02-20T08:01:58.530 INFO:teuthology.task.ansible.out:eading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137468 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-02-20T08:01:58.580 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:58 +0000 (0:00:00.087) 0:02:06.884 ***** 2021-02-20T08:01:58.853 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-02-20T08:01:58.855 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-02-20T08:01:58.891 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:58 +0000 (0:00:00.310) 0:02:07.195 ***** 2021-02-20T08:01:59.235 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-02-20T08:01:59.237 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-02-20T08:01:59.271 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:59 +0000 (0:00:00.379) 0:02:07.574 ***** 2021-02-20T08:01:59.458 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-02-20T08:01:59.459 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"cache_update_time": 1613807781, "cache_updated": false, "changed": false} 2021-02-20T08:01:59.496 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:59 +0000 (0:00:00.225) 0:02:07.800 ***** 2021-02-20T08:01:59.588 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-02-20T08:01:59.590 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-02-20T08:01:59.630 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:59 +0000 (0:00:00.134) 0:02:07.934 ***** 2021-02-20T08:01:59.638 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-02-20T08:01:59.640 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-02-20T08:01:59.687 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:59 +0000 (0:00:00.057) 0:02:07.991 ***** 2021-02-20T08:01:59.715 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2021-02-20T08:01:59.717 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:59.757 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:59 +0000 (0:00:00.068) 0:02:08.059 ***** 2021-02-20T08:01:59.772 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-02-20T08:01:59.774 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T08:01:59.812 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:59 +0000 (0:00:00.056) 0:02:08.116 ***** 2021-02-20T08:01:59.839 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-02-20T08:01:59.841 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:59.877 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:59 +0000 (0:00:00.064) 0:02:08.180 ***** 2021-02-20T08:01:59.905 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-02-20T08:01:59.907 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} TASK [testnode : Stop collectl] ************************************************ skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:01:59.966 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:01:59 +0000 (0:00:00.089) 0:02:08.269 ***** 2021-02-20T08:02:00.014 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:00 +0000 (0:00:00.047) 0:02:08.317 ***** 2021-02-20T08:02:00.021 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-02-20T08:02:00.023 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:00.073 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:00 +0000 (0:00:00.059) 0:02:08.376 ***** 2021-02-20T08:02:00.111 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2021-02-20T08:02:00.113 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:00.149 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:00 +0000 (0:00:00.074) 0:02:08.451 ***** 2021-02-20T08:02:00.186 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-02-20T08:02:00.188 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:00.224 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:00 +0000 (0:00:00.076) 0:02:08.528 ***** 2021-02-20T08:02:00.252 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T08:02:00.254 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-02-20T08:02:00.289 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:00 +0000 (0:00:00.065) 0:02:08.593 ***** 2021-02-20T08:02:00.339 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-02-20T08:02:00.340 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:00.352 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-02-20T08:02:00.354 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi003.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:00.404 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:00 +0000 (0:00:00.114) 0:02:08.707 ***** 2021-02-20T08:02:00.416 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:00.418 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} ok: [smithi003.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} skipping: [smithi003.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:00.429 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:00.431 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:00.442 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:00.443 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:00.454 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:00.465 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:00.513 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:00 +0000 (0:00:00.109) 0:02:08.817 ***** 2021-02-20T08:02:00.566 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-02-20T08:02:00.568 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-02-20T08:02:00.605 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:00 +0000 (0:00:00.091) 0:02:08.909 ***** 2021-02-20T08:02:00.735 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2021-02-20T08:02:00.737 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi003.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003372", "end": "2021-02-20 08:02:00.710581", "rc": 0, "start": "2021-02-20 08:02:00.707209", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/virtualenv/bin/teuthology-lock"]} 2021-02-20T08:02:00.787 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:00 +0000 (0:00:00.181) 0:02:09.091 ***** 2021-02-20T08:02:00.826 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:00 +0000 (0:00:00.038) 0:02:09.130 ***** 2021-02-20T08:02:00.833 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:00.875 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:00 +0000 (0:00:00.048) 0:02:09.179 ***** 2021-02-20T08:02:00.925 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-02-20T08:02:00.926 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:00.969 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:00 +0000 (0:00:00.094) 0:02:09.273 ***** 2021-02-20T08:02:00.998 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-02-20T08:02:00.999 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:01.035 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:01 +0000 (0:00:00.065) 0:02:09.339 ***** 2021-02-20T08:02:01.106 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-02-20T08:02:01.107 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sat 2021-02-20 07:59:22 UTC", "ActiveEnterTimestampMonotonic": "28419636", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target nss-lookup.target system.slice network.target remote-fs.target -.mount sysinit.target systemd-journald.socket systemd-tmpfiles-setup.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-02-20 07:59:21 UTC", "AssertTimestampMonotonic": "27906020", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-02-20 07:59:21 UTC", "ConditionTimestampMonotonic": "27906019", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2021-02-20T08:02:01.109 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1263", "ExecMainStartTimestamp": "Sat 2021-02-20 07:59:22 UTC", "ExecMainStartTimestampMonotonic": "28419619", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-02-20 07:59:21 UTC", "InactiveExitTimestampMonotonic": "27906883", "InvocationID": "a4f363e19c754b2fb506a6eb100eb82e", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2021-02-20T08:02:01.110 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128028", "LimitNPROCSoft": "128028", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128028", "LimitSIGPENDINGSoft": "128028", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1263", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2021-02-20T08:02:01.112 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 07:59:22 UTC", "StateChangeTimestampMonotonic": "28419636", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-02-20 07:59:22 UTC", "WatchdogTimestampMonotonic": "28419635", "WatchdogUSec": "0"}} 2021-02-20T08:02:01.151 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:01 +0000 (0:00:00.115) 0:02:09.454 ***** 2021-02-20T08:02:01.178 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-02-20T08:02:01.180 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:01.215 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:01 +0000 (0:00:00.064) 0:02:09.519 ***** 2021-02-20T08:02:01.245 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-02-20T08:02:01.246 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:01.281 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:01 +0000 (0:00:00.065) 0:02:09.584 ***** 2021-02-20T08:02:01.308 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-02-20T08:02:01.309 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:01.345 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:01 +0000 (0:00:00.064) 0:02:09.649 ***** 2021-02-20T08:02:01.375 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-02-20T08:02:01.377 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:01.412 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:01 +0000 (0:00:00.066) 0:02:09.716 ***** 2021-02-20T08:02:01.439 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-02-20T08:02:01.441 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:01.476 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:01 +0000 (0:00:00.064) 0:02:09.780 ***** 2021-02-20T08:02:01.504 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:01.543 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:01 +0000 (0:00:00.066) 0:02:09.846 ***** 2021-02-20T08:02:01.571 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:01.608 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:01 +0000 (0:00:00.065) 0:02:09.911 ***** 2021-02-20T08:02:01.636 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-02-20T08:02:01.637 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:01.674 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:01 +0000 (0:00:00.066) 0:02:09.978 ***** 2021-02-20T08:02:01.702 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T08:02:01.703 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:01.738 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:01 +0000 (0:00:00.063) 0:02:10.042 ***** 2021-02-20T08:02:01.767 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-02-20T08:02:01.768 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:01.803 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:01 +0000 (0:00:00.065) 0:02:10.107 ***** 2021-02-20T08:02:01.846 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:01.882 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:01 +0000 (0:00:00.078) 0:02:10.186 ***** 2021-02-20T08:02:01.912 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:01.948 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:01 +0000 (0:00:00.065) 0:02:10.252 ***** 2021-02-20T08:02:01.977 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-02-20T08:02:01.978 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:02.015 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:02 +0000 (0:00:00.067) 0:02:10.319 ***** 2021-02-20T08:02:02.023 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-02-20T08:02:02.025 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"cache_update_time": 1613807738, "cache_updated": false, "changed": false} 2021-02-20T08:02:02.063 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:02 +0000 (0:00:00.047) 0:02:10.367 ***** 2021-02-20T08:02:02.074 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-20T08:02:02.075 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:02.113 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:02 +0000 (0:00:00.049) 0:02:10.417 ***** 2021-02-20T08:02:02.121 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-02-20T08:02:02.123 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:02.167 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:02 +0000 (0:00:00.053) 0:02:10.470 ***** 2021-02-20T08:02:02.176 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-02-20T08:02:02.178 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:02.216 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:02 +0000 (0:00:00.049) 0:02:10.520 ***** 2021-02-20T08:02:02.221 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-02-20T08:02:02.223 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item={'block_used': 13756956, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226338152, 'size_available': 927081070592, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226338152, "block_size": 4096, "block_total": 240095108, "block_used": 13756956, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927081070592, "size_total": 983429562368, "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}} 2021-02-20T08:02:02.261 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:02 +0000 (0:00:00.044) 0:02:10.564 ***** 2021-02-20T08:02:02.267 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-02-20T08:02:02.269 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:02.307 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:02 +0000 (0:00:00.045) 0:02:10.610 ***** 2021-02-20T08:02:02.334 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-02-20T08:02:02.336 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:02.373 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:02 +0000 (0:00:00.066) 0:02:10.676 ***** 2021-02-20T08:02:02.391 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-02-20T08:02:02.393 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.011072", "end": "2021-02-20 08:02:02.373319", "rc": 0, "start": "2021-02-20 08:02:02.362247", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:02.430 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:02 +0000 (0:00:00.057) 0:02:10.733 ***** 2021-02-20T08:02:02.437 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-02-20T08:02:02.439 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:02.479 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:02 +0000 (0:00:00.049) 0:02:10.783 ***** 2021-02-20T08:02:02.486 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-02-20T08:02:02.487 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'block_used': 13756956, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226338152, 'size_available': 927081070592, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226338152, "block_size": 4096, "block_total": 240095108, "block_used": 13756956, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927081070592, "size_total": 983429562368, "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:02.526 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:02 +0000 (0:00:00.046) 0:02:10.830 ***** 2021-02-20T08:02:02.540 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-02-20T08:02:02.542 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:02.578 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:02 +0000 (0:00:00.052) 0:02:10.882 ***** 2021-02-20T08:02:02.585 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-02-20T08:02:02.625 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:02 +0000 (0:00:00.046) 0:02:10.928 ***** 2021-02-20T08:02:02.631 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-20T08:02:02.633 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:02.670 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:02 +0000 (0:00:00.045) 0:02:10.974 ***** 2021-02-20T08:02:02.678 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-02-20T08:02:02.680 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:02.701 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:02.743 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:02 +0000 (0:00:00.071) 0:02:11.046 ***** 2021-02-20T08:02:02.769 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-02-20T08:02:02.771 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:02.806 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:02 +0000 (0:00:00.063) 0:02:11.110 ***** 2021-02-20T08:02:02.833 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-20T08:02:02.835 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:02.870 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:02 +0000 (0:00:00.063) 0:02:11.174 ***** 2021-02-20T08:02:02.897 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-02-20T08:02:02.899 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:02.945 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:02 +0000 (0:00:00.075) 0:02:11.249 ***** 2021-02-20T08:02:02.975 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-02-20T08:02:02.977 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:03.012 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:03 +0000 (0:00:00.067) 0:02:11.316 ***** 2021-02-20T08:02:03.040 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2021-02-20T08:02:03.042 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:03.078 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:03 +0000 (0:00:00.065) 0:02:11.381 ***** 2021-02-20T08:02:03.210 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-02-20T08:02:03.212 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-02-20T08:02:03.246 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:03 +0000 (0:00:00.168) 0:02:11.550 ***** 2021-02-20T08:02:03.560 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-02-20T08:02:03.562 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-02-20T08:02:03.609 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:03 +0000 (0:00:00.362) 0:02:11.913 ***** 2021-02-20T08:02:03.638 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-02-20T08:02:03.640 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:03.680 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:03 +0000 (0:00:00.070) 0:02:11.984 ***** 2021-02-20T08:02:04.024 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-02-20T08:02:04.026 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-02-20T08:02:04.063 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:04 +0000 (0:00:00.382) 0:02:12.367 ***** 2021-02-20T08:02:04.383 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-02-20T08:02:04.385 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-02-20T08:02:04.422 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:04 +0000 (0:00:00.359) 0:02:12.726 ***** 2021-02-20T08:02:04.732 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-02-20T08:02:04.734 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-pKv8lBcvU7hEcQkz0cNO5OGbFDU2FNnKuvDRPJn56Zxm8FUUp92MOsIgWefnaZO2'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.924296", "end": "2021-02-20 08:02:04.697932", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-pKv8lBcvU7hEcQkz0cNO5OGbFDU2FNnKuvDRPJn56Zxm8FUUp92MOsIgWefnaZO2"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:02.773636", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [smithi003.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1' 2021-02-20T08:02:04.736 INFO:teuthology.task.ansible.out:, 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:04.746 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-02-20T08:02:04.748 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** skipping: [smithi003.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:04.784 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:04 +0000 (0:00:00.361) 0:02:13.088 ***** 2021-02-20T08:02:04.917 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-02-20T08:02:04.919 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T08:02:04.955 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:04 +0000 (0:00:00.171) 0:02:13.259 ***** 2021-02-20T08:02:04.982 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-02-20T08:02:04.984 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:05.018 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:05 +0000 (0:00:00.062) 0:02:13.321 ***** 2021-02-20T08:02:05.045 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-02-20T08:02:05.047 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:05.081 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:05 +0000 (0:00:00.062) 0:02:13.384 ***** 2021-02-20T08:02:05.107 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-02-20T08:02:05.109 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:05.146 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:05 +0000 (0:00:00.065) 0:02:13.450 ***** 2021-02-20T08:02:05.186 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:05.229 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:05 +0000 (0:00:00.082) 0:02:13.533 ***** 2021-02-20T08:02:05.257 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-02-20T08:02:05.259 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:05.293 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:05 +0000 (0:00:00.063) 0:02:13.596 ***** 2021-02-20T08:02:05.339 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T08:02:05.340 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-02-20T08:02:05.385 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:05 +0000 (0:00:00.092) 0:02:13.689 ***** 2021-02-20T08:02:05.436 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-02-20T08:02:05.437 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:05.447 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:05.458 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:05.469 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:05.480 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:05.491 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:05.503 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-02-20T08:02:05.505 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:05.515 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:05.525 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:05.537 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:05.548 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:05.560 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:05.561 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:05.572 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:05.615 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:05 +0000 (0:00:00.229) 0:02:13.918 ***** 2021-02-20T08:02:05.841 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-02-20T08:02:05.843 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.004570", "end": "2021-02-20 08:02:05.810009", "rc": 0, "start": "2021-02-20 08:02:05.805439", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/virtualenv/bin/teuthology-lock"]} 2021-02-20T08:02:05.908 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:05 +0000 (0:00:00.293) 0:02:14.212 ***** 2021-02-20T08:02:05.949 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:06.007 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:06 +0000 (0:00:00.098) 0:02:14.311 ***** 2021-02-20T08:02:06.015 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-02-20T08:02:06.018 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-pKv8lBcvU7hEcQkz0cNO5OGbFDU2FNnKWlnu1N7ZJgu1DZsccK5PmIG8SDGV0u9Q'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.109358", "end": "2021-02-20 08:02:05.954589", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-pKv8lBcvU7hEcQkz0cNO5OGbFDU2FNnKWlnu1N7ZJgu1DZsccK5PmIG8SDGV0u9Q"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:04.845231", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [smithi003.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', ' 2021-02-20T08:02:06.020 INFO:teuthology.task.ansible.out:vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HRN7', 'wwn-0x5000c500871a3370'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HRN7-part1', 'wwn-0x5000c500871a3370-part1'], 'uuids': ['602a9f0b-ccf6-4d5d-877a-542e0952c07a']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500871a3370', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4HRN7', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HRN7", "wwn-0x5000c500871a3370"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HRN7-part1", "wwn-0x5000c500871a3370-part1"], "labels": [], "masters": [], "uuids": ["602a9f0b-ccf6-4d5d-877a-542e0952c07a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4HRN7", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871a3370"}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:06.040 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-02-20T08:02:06.043 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:06.079 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:06 +0000 (0:00:00.071) 0:02:14.383 ***** 2021-02-20T08:02:06.108 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-02-20T08:02:06.110 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:06.149 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:06 +0000 (0:00:00.069) 0:02:14.452 ***** 2021-02-20T08:02:07.096 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-02-20T08:02:07.097 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"cache_update_time": 1613807781, "cache_updated": false, "changed": false} 2021-02-20T08:02:07.135 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:07 +0000 (0:00:00.986) 0:02:15.439 ***** 2021-02-20T08:02:07.178 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-02-20T08:02:07.180 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:07.223 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:07 +0000 (0:00:00.088) 0:02:15.527 ***** 2021-02-20T08:02:07.273 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-02-20T08:02:07.275 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item={'block_used': 13754883, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226340225, 'size_available': 927089561600, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226340225, "block_size": 4096, "block_total": 240095108, "block_used": 13754883, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927089561600, "size_total": 983429562368, "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}} 2021-02-20T08:02:07.325 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:07 +0000 (0:00:00.101) 0:02:15.628 ***** 2021-02-20T08:02:07.331 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-02-20T08:02:07.333 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-4tojP0-Gz8l-rIk6-0yji-4dfR-JRNq-UxK9sD', 'nvme-INTEL_SSDPEDMD400G4_CVFT5330009Z400BGN', 'nvme-nvme.8086-43564654353333303030395a34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.165312", "end": "2021-02-20 08:02:07.252806", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-4tojP0-Gz8l-rIk6-0yji-4dfR-JRNq-UxK9sD", "nvme-INTEL_SSDPEDMD400G4_CVFT5330009Z400BGN", "nvme-nvme.8086-43564654353333303030395a34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:06.087494", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2021-02-20T08:02:07.335 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [smithi003.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_re 2021-02-20T08:02:07.336 INFO:teuthology.task.ansible.out:ason": "Conditional result was False"} 2021-02-20T08:02:07.464 INFO:teuthology.task.ansible.out: 2021-02-20T08:02:07.466 INFO:teuthology.task.ansible.out:TASK [testnode : Compile list of non-root partitions] ************************** changed: [smithi090.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.010931", "end": "2021-02-20 08:02:07.509155", "rc": 0, "start": "2021-02-20 08:02:07.498224", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:07.501 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:07 +0000 (0:00:00.176) 0:02:15.805 ***** 2021-02-20T08:02:07.539 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-02-20T08:02:07.541 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'block_used': 13754883, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226340225, 'size_available': 927089561600, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226340225, "block_size": 4096, "block_total": 240095108, "block_used": 13754883, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927089561600, "size_total": 983429562368, "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:07.577 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:07 +0000 (0:00:00.075) 0:02:15.880 ***** 2021-02-20T08:02:07.604 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-02-20T08:02:07.640 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:07 +0000 (0:00:00.063) 0:02:15.944 ***** 2021-02-20T08:02:07.699 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-02-20T08:02:07.701 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:08.544 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-pKv8lBcvU7hEcQkz0cNO5OGbFDU2FNnKXz3ZHOKJLiVKQhnbsDwI0eFz3oqryOfY'], 'uuids': ['fc7232e2-2b9b-4a29-b053-4870ccf50116']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.125785", "end": "2021-02-20 08:02:08.518010", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-pKv8lBcvU7hEcQkz0cNO5OGbFDU2FNnKXz3ZHOKJLiVKQhnbsDwI0eFz3oqryOfY"], "labels": [], "masters": [], "uuids": ["fc7232e2-2b9b-4a29-b053-4870ccf50116"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:07.392225", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T08:02:08.556 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:08.568 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:09.796 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-pKv8lBcvU7hEcQkz0cNO5OGbFDU2FNnKOn4NhoEBQFaQyrD4GVksSyid5yueaNoQ'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.104535", "end": "2021-02-20 08:02:09.764442", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-pKv8lBcvU7hEcQkz0cNO5OGbFDU2FNnKOn4NhoEBQFaQyrD4GVksSyid5yueaNoQ"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:08.659907", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T08:02:09.807 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:10.275 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-zbMX5e4sB5QxVvMF1QGJkXfGrhash1HVVPR4gHYJQQVaCdAiWNSCATa8c48L2i0w'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:02.449883", "end": "2021-02-20 08:02:10.315026", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-zbMX5e4sB5QxVvMF1QGJkXfGrhash1HVVPR4gHYJQQVaCdAiWNSCATa8c48L2i0w"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:07.865143", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T08:02:10.290 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:10.292 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:11.041 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-pKv8lBcvU7hEcQkz0cNO5OGbFDU2FNnKYD13UJoLdhkl1vu76pzbHLcg7fmAx7vq'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.097552", "end": "2021-02-20 08:02:11.006432", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-pKv8lBcvU7hEcQkz0cNO5OGbFDU2FNnKYD13UJoLdhkl1vu76pzbHLcg7fmAx7vq"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:09.908880", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T08:02:11.092 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:11 +0000 (0:00:03.451) 0:02:19.395 ***** 2021-02-20T08:02:11.141 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-02-20T08:02:11.143 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:11.425 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-pKv8lBcvU7hEcQkz0cNO5OGbFDU2FNnKuvDRPJn56Zxm8FUUp92MOsIgWefnaZO2'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.156182", "end": "2021-02-20 08:02:11.396783", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-pKv8lBcvU7hEcQkz0cNO5OGbFDU2FNnKuvDRPJn56Zxm8FUUp92MOsIgWefnaZO2"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:11.240601", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.151388 s, 762 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.151388 s, 762 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:11.435 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:11.448 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:11.565 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-02-20T08:02:11.567 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-zbMX5e4sB5QxVvMF1QGJkXfGrhash1HVdDo6QN910cbBu9YeYI0y7BZkpblP7bTW'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.139833", "end": "2021-02-20 08:02:11.601577", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-zbMX5e4sB5QxVvMF1QGJkXfGrhash1HVdDo6QN910cbBu9YeYI0y7BZkpblP7bTW"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:10.461744", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T08:02:11.578 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-Hitachi_HUA722010CLA330_JPW9J0N20ZVK9C', 'wwn-0x5000cca3a8ce06d4'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'links': {'masters': [], 'labels': [], 'ids': ['ata-Hitachi_HUA722010CLA330_JPW9J0N20ZVK9C-part1', 'wwn-0x5000cca3a8ce06d4-part1'], 'uuids': ['602a9f0b-ccf6-4d5d-877a-542e0952c07a']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000cca3a8ce06d4', 'model': 'Hitachi HUA72201', 'serial': 'JPW9J0N20ZVK9C', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-Hitachi_HUA722010CLA330_JPW9J0N20ZVK9C", "wwn-0x5000cca3a8ce06d4"], "labels": [], "masters": [], "uuids": []}, "model": "Hitachi HUA72201", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-Hitachi_HUA722010CLA330_JPW9J0N20ZVK9C-part1", "wwn-0x5000cca3a8ce06d4-part1"], "labels": [], "masters": [], "uuids": ["602a9f0b-ccf6-4d5d-877a-542e0952c07a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "JPW9J0N20ZVK9C", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual 2021-02-20T08:02:11.580 INFO:teuthology.task.ansible.out:": 1, "wwn": "0x5000cca3a8ce06d4"}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:11.726 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-02-20T08:02:11.728 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-pKv8lBcvU7hEcQkz0cNO5OGbFDU2FNnKWlnu1N7ZJgu1DZsccK5PmIG8SDGV0u9Q'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.155564", "end": "2021-02-20 08:02:11.706665", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-pKv8lBcvU7hEcQkz0cNO5OGbFDU2FNnKWlnu1N7ZJgu1DZsccK5PmIG8SDGV0u9Q"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:11.551101", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.152536 s, 756 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.152536 s, 756 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:11.751 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HRN7', 'wwn-0x5000c500871a3370'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HRN7-part1', 'wwn-0x5000c500871a3370-part1'], 'uuids': ['602a9f0b-ccf6-4d5d-877a-542e0952c07a']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500871a3370', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4HRN7', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HRN7", "wwn-0x5000c500871a3370"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HRN7-part1", "wwn-0x5000c500871a3370-part1"], "labels": [], "masters": [], "uuids": ["602a9f0b-ccf6-4d5d-877a-542e0952c07a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4HRN7", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871a3370"}}, "skip_reason": 2021-02-20T08:02:11.757 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2021-02-20T08:02:11.941 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-4tojP0-Gz8l-rIk6-0yji-4dfR-JRNq-UxK9sD', 'nvme-INTEL_SSDPEDMD400G4_CVFT5330009Z400BGN', 'nvme-nvme.8086-43564654353333303030395a34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.055886", "end": "2021-02-20 08:02:11.908787", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-4tojP0-Gz8l-rIk6-0yji-4dfR-JRNq-UxK9sD", "nvme-INTEL_SSDPEDMD400G4_CVFT5330009Z400BGN", "nvme-nvme.8086-43564654353333303030395a34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:11.852901", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0510322 s, 2.3 GB/s", "std 2021-02-20T08:02:11.944 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0510322 s, 2.3 GB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:11.944 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:11.956 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:12.253 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-pKv8lBcvU7hEcQkz0cNO5OGbFDU2FNnKXz3ZHOKJLiVKQhnbsDwI0eFz3oqryOfY'], 'uuids': ['fc7232e2-2b9b-4a29-b053-4870ccf50116']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.161781", "end": "2021-02-20 08:02:12.218557", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-pKv8lBcvU7hEcQkz0cNO5OGbFDU2FNnKXz3ZHOKJLiVKQhnbsDwI0eFz3oqryOfY"], "labels": [], "masters": [], "uuids": ["fc7232e2-2b9b-4a29-b053-4870ccf50116"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:12.056776", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.158296 s, 729 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.158296 s, 729 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:12.265 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:12.267 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:12.550 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-pKv8lBcvU7hEcQkz0cNO5OGbFDU2FNnKOn4NhoEBQFaQyrD4GVksSyid5yueaNoQ'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.158739", "end": "2021-02-20 08:02:12.520303", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-pKv8lBcvU7hEcQkz0cNO5OGbFDU2FNnKOn4NhoEBQFaQyrD4GVksSyid5yueaNoQ"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:12.361564", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.154094 s, 749 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.154094 s, 749 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:12.572 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:12.857 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-pKv8lBcvU7hEcQkz0cNO5OGbFDU2FNnKYD13UJoLdhkl1vu76pzbHLcg7fmAx7vq'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.157419", "end": "2021-02-20 08:02:12.826310", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-pKv8lBcvU7hEcQkz0cNO5OGbFDU2FNnKYD13UJoLdhkl1vu76pzbHLcg7fmAx7vq"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:12.668891", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.152517 s, 756 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.152517 s, 756 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:12.909 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:12 +0000 (0:00:01.816) 0:02:21.212 ***** 2021-02-20T08:02:12.916 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-02-20T08:02:12.919 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-KQktEg-i8It-1h4D-Yjjz-rv5Z-DW9I-QDEPMa', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400UK400BGN', 'nvme-nvme.8086-50484654363230343030554b34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.169335", "end": "2021-02-20 08:02:12.904552", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-KQktEg-i8It-1h4D-Yjjz-rv5Z-DW9I-QDEPMa", "nvme-INTEL_SSDPEDMD400G4_PHFT620400UK400BGN", "nvme-nvme.8086-50484654363230343030554b34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:11.735217", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2021-02-20T08:02:12.921 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [smithi090.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_re 2021-02-20T08:02:12.923 INFO:teuthology.task.ansible.out:ason": "Conditional result was False"} 2021-02-20T08:02:13.216 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** changed: [smithi003.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.168741", "end": "2021-02-20 08:02:13.188744", "rc": 0, "start": "2021-02-20 08:02:13.020003", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:13.253 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:13 +0000 (0:00:00.344) 0:02:21.557 ***** 2021-02-20T08:02:13.453 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-02-20T08:02:13.455 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.070881", "end": "2021-02-20 08:02:13.435035", "rc": 0, "start": "2021-02-20 08:02:13.364154", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:13.492 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:13 +0000 (0:00:00.239) 0:02:21.796 ***** 2021-02-20T08:02:13.521 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-02-20T08:02:13.560 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:13 +0000 (0:00:00.068) 0:02:21.864 ***** 2021-02-20T08:02:13.598 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:13.934 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:13 +0000 (0:00:00.083) 0:02:21.948 ***** TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Saturday 20 February 2021 08:02:13 +0000 (0:00:00.099) 0:02:22.047 ***** TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Saturday 20 February 2021 08:02:13 +0000 (0:00:00.067) 0:02:22.115 ***** TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Saturday 20 February 2021 08:02:13 +0000 (0:00:00.069) 0:02:22.184 ***** 2021-02-20T08:02:14.015 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-02-20T08:02:14.017 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T08:02:14.052 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:14 +0000 (0:00:00.171) 0:02:22.355 ***** 2021-02-20T08:02:14.100 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-02-20T08:02:14.102 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'block_used': 13756956, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226338152, 'size_available': 927081070592, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226338152, "block_size": 4096, "block_total": 240095108, "block_used": 13756956, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927081070592, "size_total": 983429562368, "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:14.140 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:14 +0000 (0:00:00.087) 0:02:22.443 ***** 2021-02-20T08:02:14.157 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-02-20T08:02:14.159 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-zbMX5e4sB5QxVvMF1QGJkXfGrhash1HVQfTFtxOzfGZt1tzz1fjRNdQSRpjfvDug'], 'uuids': ['75352f00-f8ae-4acb-8d5f-338a1f175135']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.143846", "end": "2021-02-20 08:02:14.193478", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-zbMX5e4sB5QxVvMF1QGJkXfGrhash1HVQfTFtxOzfGZt1tzz1fjRNdQSRpjfvDug"], "labels": [], "masters": [], "uuids": ["75352f00-f8ae-4acb-8d5f-338a1f175135"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:13.049632", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T08:02:14.169 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:14.181 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-02-20T08:02:14.182 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Zap all non-root disks] *************************************** skipping: [smithi090.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:14.222 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:14 +0000 (0:00:00.082) 0:02:22.526 ***** 2021-02-20T08:02:14.260 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:14.296 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:14 +0000 (0:00:00.073) 0:02:22.600 ***** 2021-02-20T08:02:14.323 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T08:02:14.325 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:14.361 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:14 +0000 (0:00:00.065) 0:02:22.665 ***** 2021-02-20T08:02:14.388 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-02-20T08:02:14.390 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:14.426 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:14 +0000 (0:00:00.064) 0:02:22.729 ***** 2021-02-20T08:02:14.454 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-02-20T08:02:14.456 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:14.489 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:14 +0000 (0:00:00.063) 0:02:22.793 ***** 2021-02-20T08:02:14.517 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-02-20T08:02:14.519 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:14.553 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:14 +0000 (0:00:00.064) 0:02:22.857 ***** 2021-02-20T08:02:15.135 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2021-02-20T08:02:15.137 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-02-20T08:02:15.171 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:15 +0000 (0:00:00.617) 0:02:23.475 ***** 2021-02-20T08:02:15.450 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-02-20T08:02:15.452 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-zbMX5e4sB5QxVvMF1QGJkXfGrhash1HVcN8x28qMEWqg7MPBL9qratPpZhB50cIx'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.149825", "end": "2021-02-20 08:02:15.483999", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-zbMX5e4sB5QxVvMF1QGJkXfGrhash1HVcN8x28qMEWqg7MPBL9qratPpZhB50cIx"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:14.334174", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T08:02:15.462 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:15.750 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-02-20T08:02:15.752 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T08:02:15.981 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T08:02:16.212 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T08:02:16.460 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T08:02:16.713 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T08:02:16.754 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:16 +0000 (0:00:01.583) 0:02:25.058 ***** 2021-02-20T08:02:16.763 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-02-20T08:02:16.765 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-zbMX5e4sB5QxVvMF1QGJkXfGrhash1HV7jNyfJxBR1wTpub2AiDXRYRvteyUynot'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.157019", "end": "2021-02-20 08:02:16.783897", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-zbMX5e4sB5QxVvMF1QGJkXfGrhash1HV7jNyfJxBR1wTpub2AiDXRYRvteyUynot"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:15.626878", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T08:02:16.818 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:16 +0000 (0:00:00.063) 0:02:25.122 ***** 2021-02-20T08:02:16.869 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-02-20T08:02:16.871 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:16.882 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** changed: [smithi003.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-02-20T08:02:16.920 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:16 +0000 (0:00:00.101) 0:02:25.224 ***** 2021-02-20T08:02:17.076 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-02-20T08:02:17.078 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T08:02:17.151 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-02-20T08:02:17.153 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-zbMX5e4sB5QxVvMF1QGJkXfGrhash1HVVPR4gHYJQQVaCdAiWNSCATa8c48L2i0w'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.158216", "end": "2021-02-20 08:02:17.186569", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-zbMX5e4sB5QxVvMF1QGJkXfGrhash1HVVPR4gHYJQQVaCdAiWNSCATa8c48L2i0w"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:17.028353", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.153548 s, 751 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.153548 s, 751 MB/s"], "stdout": "", "stdout_lines": []} skipping: [smithi090.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": 2021-02-20T08:02:17.155 INFO:teuthology.task.ansible.out:{"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:17.164 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:17.198 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T08:02:17.311 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T08:02:17.428 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T08:02:17.430 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-zbMX5e4sB5QxVvMF1QGJkXfGrhash1HVdDo6QN910cbBu9YeYI0y7BZkpblP7bTW'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.157872", "end": "2021-02-20 08:02:17.465187", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-zbMX5e4sB5QxVvMF1QGJkXfGrhash1HVdDo6QN910cbBu9YeYI0y7BZkpblP7bTW"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:17.307315", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.153276 s, 753 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.153276 s, 753 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:17.442 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-Hitachi_HUA722010CLA330_JPW9J0N20ZVK9C', 'wwn-0x5000cca3a8ce06d4'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'links': {'masters': [], 'labels': [], 'ids': ['ata-Hitachi_HUA722010CLA330_JPW9J0N20ZVK9C-part1', 'wwn-0x5000cca3a8ce06d4-part1'], 'uuids': ['602a9f0b-ccf6-4d5d-877a-542e0952c07a']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000cca3a8ce06d4', 'model': 'Hitachi HUA72201', 'serial': 'JPW9J0N20ZVK9C', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-Hitachi_HUA722010CLA330_JPW9J0N20ZVK9C", "wwn-0x5000cca3a8ce06d4"], "labels": [], "masters": [], "uuids": []}, "model": "Hitachi HUA72201", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-Hitachi_HUA722010CLA330_JPW9J0N20ZVK9C-part1", "wwn-0x5000cca3a8ce06d4-part1"], "labels": [], "masters": [], "uuids": ["602a9f0b-ccf6-4d5d-877a-542e0952c07a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "JPW9J0N20ZVK9C", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual 2021-02-20T08:02:17.444 INFO:teuthology.task.ansible.out:": 1, "wwn": "0x5000cca3a8ce06d4"}}, "skip_reason": "Conditional result was False"} TASK [testnode : Write /scratch_devs] ****************************************** skipping: [smithi003.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:17.485 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:17 +0000 (0:00:00.564) 0:02:25.789 ***** 2021-02-20T08:02:17.514 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-02-20T08:02:17.516 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:17.552 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:17 +0000 (0:00:00.067) 0:02:25.856 ***** 2021-02-20T08:02:17.580 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-02-20T08:02:17.581 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:17.619 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:17 +0000 (0:00:00.066) 0:02:25.922 ***** 2021-02-20T08:02:17.626 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-02-20T08:02:17.627 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-KQktEg-i8It-1h4D-Yjjz-rv5Z-DW9I-QDEPMa', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400UK400BGN', 'nvme-nvme.8086-50484654363230343030554b34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.050888", "end": "2021-02-20 08:02:17.648522", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-KQktEg-i8It-1h4D-Yjjz-rv5Z-DW9I-QDEPMa", "nvme-INTEL_SSDPEDMD400G4_PHFT620400UK400BGN", "nvme-nvme.8086-50484654363230343030554b34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:17.597634", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0462983 s, 2.5 GB/s", "std 2021-02-20T08:02:17.629 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0462983 s, 2.5 GB/s"], "stdout": "", "stdout_lines": []} skipping: [smithi090.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:17.641 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:17.757 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-02-20T08:02:17.759 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T08:02:17.795 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:17 +0000 (0:00:00.176) 0:02:26.099 ***** 2021-02-20T08:02:17.833 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-02-20T08:02:17.835 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-02-20T08:02:17.871 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:17 +0000 (0:00:00.076) 0:02:26.175 ***** 2021-02-20T08:02:17.903 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-02-20T08:02:17.905 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-zbMX5e4sB5QxVvMF1QGJkXfGrhash1HVQfTFtxOzfGZt1tzz1fjRNdQSRpjfvDug'], 'uuids': ['75352f00-f8ae-4acb-8d5f-338a1f175135']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.145548", "end": "2021-02-20 08:02:17.936068", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-zbMX5e4sB5QxVvMF1QGJkXfGrhash1HVQfTFtxOzfGZt1tzz1fjRNdQSRpjfvDug"], "labels": [], "masters": [], "uuids": ["75352f00-f8ae-4acb-8d5f-338a1f175135"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:17.790520", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.141911 s, 813 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.141911 s, 813 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:17.915 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:17.927 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:18.212 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-zbMX5e4sB5QxVvMF1QGJkXfGrhash1HVcN8x28qMEWqg7MPBL9qratPpZhB50cIx'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.155327", "end": "2021-02-20 08:02:18.241729", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-zbMX5e4sB5QxVvMF1QGJkXfGrhash1HVcN8x28qMEWqg7MPBL9qratPpZhB50cIx"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:18.086402", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.152087 s, 758 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.152087 s, 758 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:18.224 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:18.456 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [smithi003.front.sepia.ceph.com] => {"changed": true} 2021-02-20T08:02:18.499 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:18 +0000 (0:00:00.627) 0:02:26.802 ***** 2021-02-20T08:02:18.532 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-02-20T08:02:18.533 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-zbMX5e4sB5QxVvMF1QGJkXfGrhash1HV7jNyfJxBR1wTpub2AiDXRYRvteyUynot'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.164320", "end": "2021-02-20 08:02:18.566206", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-zbMX5e4sB5QxVvMF1QGJkXfGrhash1HV7jNyfJxBR1wTpub2AiDXRYRvteyUynot"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 08:02:18.401886", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.159654 s, 722 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.159654 s, 722 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:18.605 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:18 +0000 (0:00:00.106) 0:02:26.908 ***** 2021-02-20T08:02:18.930 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-02-20T08:02:18.932 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.173379", "end": "2021-02-20 08:02:18.964585", "rc": 0, "start": "2021-02-20 08:02:18.791206", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:18.967 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:18 +0000 (0:00:00.361) 0:02:27.270 ***** 2021-02-20T08:02:19.070 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2021-02-20T08:02:19.072 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-02-20T08:02:19.114 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:19 +0000 (0:00:00.147) 0:02:27.418 ***** 2021-02-20T08:02:19.142 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2021-02-20T08:02:19.144 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:19.191 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:19 +0000 (0:00:00.077) 0:02:27.495 ***** 2021-02-20T08:02:19.199 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-02-20T08:02:19.200 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.047825", "end": "2021-02-20 08:02:19.199357", "rc": 0, "start": "2021-02-20 08:02:19.151532", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:19.237 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:19 +0000 (0:00:00.045) 0:02:27.541 ***** 2021-02-20T08:02:19.244 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2021-02-20T08:02:19.246 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:19.286 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:19 +0000 (0:00:00.048) 0:02:27.589 ***** 2021-02-20T08:02:19.295 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-02-20T08:02:19.338 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:19 +0000 (0:00:00.052) 0:02:27.642 ***** 2021-02-20T08:02:19.346 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:19.407 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:19 +0000 (0:00:00.068) 0:02:27.710 ***** 2021-02-20T08:02:19.425 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-02-20T08:02:19.430 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:19.474 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:19 +0000 (0:00:00.067) 0:02:27.778 ***** 2021-02-20T08:02:19.481 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-02-20T08:02:19.483 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:19.519 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:19 +0000 (0:00:00.045) 0:02:27.823 ***** 2021-02-20T08:02:19.528 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-02-20T08:02:19.530 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:19.569 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:19 +0000 (0:00:00.049) 0:02:27.872 ***** 2021-02-20T08:02:19.598 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-02-20T08:02:19.600 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:19.636 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:19 +0000 (0:00:00.067) 0:02:27.940 ***** 2021-02-20T08:02:19.665 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:19.703 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:19 +0000 (0:00:00.065) 0:02:28.006 ***** 2021-02-20T08:02:19.709 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-02-20T08:02:19.710 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-02-20T08:02:19.757 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:19 +0000 (0:00:00.054) 0:02:28.060 ***** 2021-02-20T08:02:19.831 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-02-20T08:02:19.833 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T08:02:19.870 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:19 +0000 (0:00:00.113) 0:02:28.173 ***** 2021-02-20T08:02:19.921 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-02-20T08:02:19.922 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} TASK [testnode : Set root disk] ************************************************ skipping: [smithi090.front.sepia.ceph.com] => (item={'block_used': 13754883, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226340225, 'size_available': 927089561600, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226340225, "block_size": 4096, "block_total": 240095108, "block_used": 13754883, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927089561600, "size_total": 983429562368, "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:19.960 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:19 +0000 (0:00:00.089) 0:02:28.263 ***** 2021-02-20T08:02:20.023 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:20 +0000 (0:00:00.063) 0:02:28.327 ***** 2021-02-20T08:02:20.053 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-02-20T08:02:20.055 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:20.091 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:20 +0000 (0:00:00.067) 0:02:28.395 ***** 2021-02-20T08:02:20.120 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T08:02:20.122 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:20.160 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:20 +0000 (0:00:00.069) 0:02:28.464 ***** 2021-02-20T08:02:20.187 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T08:02:20.189 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:20.249 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:20 +0000 (0:00:00.088) 0:02:28.553 ***** 2021-02-20T08:02:20.290 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-02-20T08:02:20.291 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:20.330 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:20 +0000 (0:00:00.080) 0:02:28.633 ***** 2021-02-20T08:02:20.337 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-02-20T08:02:20.338 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-02-20T08:02:20.378 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:20 +0000 (0:00:00.047) 0:02:28.681 ***** 2021-02-20T08:02:20.385 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-02-20T08:02:20.387 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:20.425 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:20 +0000 (0:00:00.047) 0:02:28.729 ***** 2021-02-20T08:02:20.453 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-02-20T08:02:20.454 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:20.493 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:20 +0000 (0:00:00.068) 0:02:28.797 ***** 2021-02-20T08:02:20.706 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-02-20T08:02:20.708 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-02-20T08:02:20.745 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:20 +0000 (0:00:00.251) 0:02:29.048 ***** 2021-02-20T08:02:20.849 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2021-02-20T08:02:20.851 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-02-20T08:02:20.893 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:20 +0000 (0:00:00.147) 0:02:29.196 ***** 2021-02-20T08:02:21.195 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T08:02:21.427 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T08:02:21.661 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T08:02:21.683 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-02-20T08:02:21.685 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"cache_update_time": 1613807738, "cache_updated": false, "changed": false} 2021-02-20T08:02:21.720 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:21 +0000 (0:00:00.827) 0:02:30.024 ***** 2021-02-20T08:02:21.887 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-02-20T08:02:21.889 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T08:02:21.900 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-02-20T08:02:21.902 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.038918", "end": "2021-02-20 08:02:21.877064", "rc": 0, "start": "2021-02-20 08:02:21.838146", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2021-02-20T08:02:21.950 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:21 +0000 (0:00:00.229) 0:02:30.253 ***** 2021-02-20T08:02:21.993 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-02-20T08:02:21.995 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:22.038 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:22 +0000 (0:00:00.088) 0:02:30.342 ***** 2021-02-20T08:02:22.077 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2021-02-20T08:02:22.079 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:22.127 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:22 +0000 (0:00:00.088) 0:02:30.430 ***** 2021-02-20T08:02:22.138 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-02-20T08:02:22.139 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T08:02:22.193 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:22 +0000 (0:00:00.066) 0:02:30.497 ***** 2021-02-20T08:02:22.328 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-02-20T08:02:22.330 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-02-20T08:02:22.367 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:22 +0000 (0:00:00.174) 0:02:30.671 ***** 2021-02-20T08:02:22.538 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-02-20T08:02:22.540 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T08:02:22.665 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T08:02:22.781 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T08:02:22.908 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T08:02:22.920 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:22.960 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:22 +0000 (0:00:00.592) 0:02:31.263 ***** 2021-02-20T08:02:22.987 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-02-20T08:02:22.989 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:23.027 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:23 +0000 (0:00:00.067) 0:02:31.331 ***** 2021-02-20T08:02:23.054 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:23.056 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** ok: [smithi003.front.sepia.ceph.com] => {"cache_update_time": 1613807738, "cache_updated": false, "changed": false} 2021-02-20T08:02:23.093 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:23 +0000 (0:00:00.065) 0:02:31.396 ***** 2021-02-20T08:02:23.131 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:23 +0000 (0:00:00.037) 0:02:31.434 ***** 2021-02-20T08:02:23.137 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2021-02-20T08:02:23.139 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:23.181 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:23 +0000 (0:00:00.049) 0:02:31.484 ***** 2021-02-20T08:02:23.275 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-02-20T08:02:23.277 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T08:02:23.313 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:23 +0000 (0:00:00.131) 0:02:31.616 ***** 2021-02-20T08:02:23.363 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-02-20T08:02:23.365 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-02-20T08:02:23.399 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:23 +0000 (0:00:00.086) 0:02:31.703 ***** 2021-02-20T08:02:23.533 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-02-20T08:02:23.535 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-02-20T08:02:23.570 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:23 +0000 (0:00:00.170) 0:02:31.874 ***** 2021-02-20T08:02:23.598 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-02-20T08:02:23.601 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:23.653 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:23 +0000 (0:00:00.083) 0:02:31.957 ***** 2021-02-20T08:02:23.879 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-02-20T08:02:23.881 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-02-20 07:58:44 UTC", "ActiveEnterTimestampMonotonic": "24691911", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sysinit.target systemd-journald.socket systemd-tmpfiles-setup.service -.mount system.slice basic.target network.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-02-20 07:58:37 UTC", "AssertTimestampMonotonic": "18160055", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-02-20 07:58:37 UTC", "ConditionTimestampMonotonic": "18160055", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2021-02-20T08:02:23.883 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1792", "ExecMainStartTimestamp": "Sat 2021-02-20 07:58:44 UTC", "ExecMainStartTimestampMonotonic": "24691898", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-02-20 07:58:37 UTC", "InactiveExitTimestampMonotonic": "18160836", "InvocationID": "30dcc6a331db4ebba9187e8d77bc9827", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128053", "LimitNPROCSoft": "128053", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2021-02-20T08:02:23.884 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128053", "LimitSIGPENDINGSoft": "128053", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1792", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 07:58:44 UTC", "S 2021-02-20T08:02:23.886 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "24691911", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-02-20 07:58:44 UTC", "WatchdogTimestampMonotonic": "24691909", "WatchdogUSec": "0"}} 2021-02-20T08:02:23.921 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:23 +0000 (0:00:00.267) 0:02:32.224 ***** 2021-02-20T08:02:23.948 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T08:02:23.950 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-02-20T08:02:23.986 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:23 +0000 (0:00:00.065) 0:02:32.290 ***** 2021-02-20T08:02:24.014 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T08:02:24.016 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:24.051 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:24 +0000 (0:00:00.064) 0:02:32.355 ***** 2021-02-20T08:02:24.079 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-02-20T08:02:24.081 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:24.117 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:24 +0000 (0:00:00.066) 0:02:32.421 ***** 2021-02-20T08:02:24.652 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2021-02-20T08:02:24.654 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"changed": true} 2021-02-20T08:02:24.691 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:24 +0000 (0:00:00.573) 0:02:32.994 ***** 2021-02-20T08:02:24.910 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2021-02-20T08:02:24.912 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-02-20T08:02:24.948 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:24 +0000 (0:00:00.256) 0:02:33.251 ***** 2021-02-20T08:02:24.976 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2021-02-20T08:02:24.978 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:25.013 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:25 +0000 (0:00:00.065) 0:02:33.317 ***** 2021-02-20T08:02:25.021 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-02-20T08:02:25.023 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"cache_update_time": 1613807738, "cache_updated": false, "changed": false} 2021-02-20T08:02:25.063 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:25 +0000 (0:00:00.049) 0:02:33.366 ***** 2021-02-20T08:02:25.070 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:25.108 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:25 +0000 (0:00:00.045) 0:02:33.412 ***** 2021-02-20T08:02:25.115 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-02-20T08:02:25.249 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:25.285 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:25 +0000 (0:00:00.176) 0:02:33.589 ***** 2021-02-20T08:02:25.292 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-02-20T08:02:25.294 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:25.336 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:25 +0000 (0:00:00.050) 0:02:33.640 ***** 2021-02-20T08:02:25.365 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-02-20T08:02:25.367 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:25.412 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:25 +0000 (0:00:00.076) 0:02:33.716 ***** 2021-02-20T08:02:25.431 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-02-20T08:02:25.433 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-02-20T08:02:25.479 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:25 +0000 (0:00:00.066) 0:02:33.783 ***** 2021-02-20T08:02:25.551 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-02-20T08:02:25.553 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-02-20T08:02:25.590 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:25 +0000 (0:00:00.110) 0:02:33.894 ***** 2021-02-20T08:02:25.724 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-02-20T08:02:25.726 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T08:02:25.761 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:25 +0000 (0:00:00.170) 0:02:34.065 ***** 2021-02-20T08:02:25.814 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-02-20T08:02:25.816 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-02-20T08:02:25.854 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:25 +0000 (0:00:00.092) 0:02:34.157 ***** 2021-02-20T08:02:26.093 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-02-20T08:02:26.095 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-02-20T08:02:26.133 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:26 +0000 (0:00:00.278) 0:02:34.436 ***** 2021-02-20T08:02:26.491 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-02-20T08:02:26.494 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-02-20T08:02:26.533 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:26 +0000 (0:00:00.400) 0:02:34.836 ***** 2021-02-20T08:02:26.802 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2021-02-20T08:02:26.804 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-02-20T08:02:26.849 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:26 +0000 (0:00:00.315) 0:02:35.152 ***** 2021-02-20T08:02:27.378 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-02-20T08:02:27.380 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false} 2021-02-20T08:02:27.415 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:27 +0000 (0:00:00.566) 0:02:35.719 ***** 2021-02-20T08:02:27.442 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-02-20T08:02:27.444 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} 2021-02-20T08:02:27.478 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:27 +0000 (0:00:00.063) 0:02:35.782 ***** 2021-02-20T08:02:27.486 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-02-20T08:02:27.488 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"cache_update_time": 1613807781, "cache_updated": false, "changed": false} 2021-02-20T08:02:27.532 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:27 +0000 (0:00:00.053) 0:02:35.835 ***** 2021-02-20T08:02:27.645 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-02-20T08:02:27.647 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1613807633.0665836, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1613807633.0625837, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 746, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1613807633.0625837, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 67, "uid": 0, "version": "3375002756", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-02-20T08:02:27.684 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:27 +0000 (0:00:00.152) 0:02:35.988 ***** 2021-02-20T08:02:27.723 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-02-20T08:02:27.725 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.051247", "end": "2021-02-20 08:02:27.762691", "rc": 0, "start": "2021-02-20 08:02:27.711444", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2021-02-20T08:02:27.768 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:27 +0000 (0:00:00.083) 0:02:36.072 ***** 2021-02-20T08:02:27.806 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-02-20T08:02:27.808 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:27.842 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:27 +0000 (0:00:00.073) 0:02:36.146 ***** 2021-02-20T08:02:27.870 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2021-02-20T08:02:27.873 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:27.916 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:27 +0000 (0:00:00.073) 0:02:36.220 ***** 2021-02-20T08:02:27.967 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-02-20T08:02:27.968 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-02-20T08:02:28.006 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:28 +0000 (0:00:00.090) 0:02:36.310 ***** 2021-02-20T08:02:28.045 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-02-20T08:02:28.047 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-02-20T08:02:28.094 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:28 +0000 (0:00:00.087) 0:02:36.397 ***** 2021-02-20T08:02:28.123 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-02-20T08:02:28.125 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:28.169 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:28 +0000 (0:00:00.074) 0:02:36.472 ***** 2021-02-20T08:02:28.315 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** changed: [smithi003.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-02-20T08:02:28.351 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:28 +0000 (0:00:00.182) 0:02:36.655 ***** 2021-02-20T08:02:28.484 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-02-20T08:02:28.486 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-02-20T08:02:28.528 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:28 +0000 (0:00:00.177) 0:02:36.832 ***** 2021-02-20T08:02:28.557 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:28.594 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:28 +0000 (0:00:00.065) 0:02:36.897 ***** 2021-02-20T08:02:28.652 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-02-20T08:02:28.654 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} skipping: [smithi003.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:28.664 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:28.666 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:28.677 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:28.742 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:28 +0000 (0:00:00.147) 0:02:37.045 ***** 2021-02-20T08:02:28.785 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-02-20T08:02:28.787 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:28.797 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:28.808 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:28.819 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:28.821 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} TASK [testnode : Install ntp package on apt based systems.] ******************** ok: [smithi090.front.sepia.ceph.com] => {"cache_update_time": 1613807781, "cache_updated": false, "changed": false} 2021-02-20T08:02:28.859 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:28 +0000 (0:00:00.117) 0:02:37.163 ***** 2021-02-20T08:02:28.901 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:28 +0000 (0:00:00.041) 0:02:37.204 ***** 2021-02-20T08:02:28.907 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T08:02:28.909 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:28.957 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:28 +0000 (0:00:00.056) 0:02:37.261 ***** 2021-02-20T08:02:28.967 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2021-02-20T08:02:28.969 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:29.017 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:29 +0000 (0:00:00.060) 0:02:37.321 ***** 2021-02-20T08:02:29.026 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-02-20T08:02:29.027 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:29.065 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:29 +0000 (0:00:00.047) 0:02:37.369 ***** 2021-02-20T08:02:29.093 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-02-20T08:02:29.094 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:29.162 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:29 +0000 (0:00:00.096) 0:02:37.465 ***** 2021-02-20T08:02:29.204 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-02-20T08:02:29.206 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:29.257 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:29 +0000 (0:00:00.095) 0:02:37.561 ***** 2021-02-20T08:02:29.286 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:29.334 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:29 +0000 (0:00:00.077) 0:02:37.638 ***** 2021-02-20T08:02:29.343 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-02-20T08:02:29.345 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-02-20T08:02:29.396 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:29 +0000 (0:00:00.062) 0:02:37.700 ***** 2021-02-20T08:02:29.404 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-02-20T08:02:29.406 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:29.452 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:29 +0000 (0:00:00.056) 0:02:37.756 ***** 2021-02-20T08:02:29.462 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-02-20T08:02:29.464 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:29.506 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:29 +0000 (0:00:00.053) 0:02:37.809 ***** 2021-02-20T08:02:29.512 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-02-20T08:02:29.515 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:29.558 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:29 +0000 (0:00:00.052) 0:02:37.862 ***** 2021-02-20T08:02:29.599 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-02-20T08:02:29.601 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:29.648 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:29 +0000 (0:00:00.088) 0:02:37.951 ***** 2021-02-20T08:02:29.744 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-02-20T08:02:29.745 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-02-20 07:59:27 UTC", "ActiveEnterTimestampMonotonic": "33980813", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service system.slice basic.target -.mount sysinit.target network.target systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-02-20 07:59:21 UTC", "AssertTimestampMonotonic": "27476042", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-02-20 07:59:21 UTC", "ConditionTimestampMonotonic": "27476041", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2021-02-20T08:02:29.747 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1639", "ExecMainStartTimestamp": "Sat 2021-02-20 07:59:27 UTC", "ExecMainStartTimestampMonotonic": "33980747", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-02-20 07:59:21 UTC", "InactiveExitTimestampMonotonic": "27476890", "InvocationID": "8b9ffef38e3d42019e6ed6da8bf7dc98", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128028", "LimitNPROCSoft": "128028", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2021-02-20T08:02:29.748 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128028", "LimitSIGPENDINGSoft": "128028", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1639", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 07:59:27 UTC", "S 2021-02-20T08:02:29.750 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "33980813", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-02-20 07:59:27 UTC", "WatchdogTimestampMonotonic": "33980805", "WatchdogUSec": "0"}} 2021-02-20T08:02:29.794 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:29 +0000 (0:00:00.146) 0:02:38.097 ***** 2021-02-20T08:02:29.831 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T08:02:29.833 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-02-20T08:02:29.893 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:29 +0000 (0:00:00.099) 0:02:38.196 ***** 2021-02-20T08:02:29.902 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-02-20T08:02:29.905 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.009101", "end": "2021-02-20 08:02:29.806834", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-02-20 08:02:29.797733", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:29.937 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T08:02:29.939 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:30.003 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:30 +0000 (0:00:00.110) 0:02:38.307 ***** 2021-02-20T08:02:30.010 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-02-20T08:02:30.012 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006221", "end": "2021-02-20 08:02:29.935748", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-02-20 08:02:29.929527", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:30.033 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-02-20T08:02:30.035 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:30.077 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:30 +0000 (0:00:00.073) 0:02:38.380 ***** 2021-02-20T08:02:30.104 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-02-20T08:02:30.107 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005575", "end": "2021-02-20 08:02:30.072548", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-02-20 08:02:30.066973", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:30.244 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.022412", "end": "2021-02-20 08:02:30.209285", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-02-20 08:02:30.186873", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:30.286 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:30 +0000 (0:00:00.208) 0:02:38.589 ***** 2021-02-20T08:02:30.315 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-02-20T08:02:30.382 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:30 +0000 (0:00:00.095) 0:02:38.685 ***** 2021-02-20T08:02:30.528 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-02-20T08:02:30.530 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-02-20T08:02:31.018 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-02-20T08:02:31.020 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"cache_update_time": 1613807781, "cache_updated": false, "changed": false} 2021-02-20T08:02:31.053 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:31 +0000 (0:00:00.671) 0:02:39.357 ***** 2021-02-20T08:02:31.082 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-02-20T08:02:31.084 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:31.120 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:31 +0000 (0:00:00.066) 0:02:39.423 ***** 2021-02-20T08:02:31.263 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-02-20T08:02:31.265 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-02-20T08:02:31.300 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:31 +0000 (0:00:00.180) 0:02:39.604 ***** 2021-02-20T08:02:31.659 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-02-20T08:02:31.696 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:31 +0000 (0:00:00.395) 0:02:39.999 ***** 2021-02-20T08:02:32.522 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2021-02-20T08:02:32.524 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-02-20T08:02:32.560 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:32 +0000 (0:00:00.864) 0:02:40.864 ***** 2021-02-20T08:02:33.089 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-02-20T08:02:33.091 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false} 2021-02-20T08:02:33.132 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:33 +0000 (0:00:00.571) 0:02:41.436 ***** 2021-02-20T08:02:33.160 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-02-20T08:02:33.162 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2021-02-20T08:02:33.198 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:33 +0000 (0:00:00.065) 0:02:41.501 ***** 2021-02-20T08:02:33.341 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-02-20T08:02:33.342 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1613807724.2519786, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1613807724.2479787, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 746, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1613807724.2479787, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "3375002756", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-02-20T08:02:33.391 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:33 +0000 (0:00:00.192) 0:02:41.694 ***** 2021-02-20T08:02:33.551 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-02-20T08:02:33.553 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-02-20T08:02:33.590 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:33 +0000 (0:00:00.199) 0:02:41.894 ***** 2021-02-20T08:02:33.620 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-02-20T08:02:33.621 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-02-20T08:02:33.668 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:33 +0000 (0:00:00.077) 0:02:41.972 ***** 2021-02-20T08:02:33.721 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:33.783 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:33 +0000 (0:00:00.114) 0:02:42.086 ***** 2021-02-20T08:02:33.930 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2021-02-20T08:02:33.932 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-02-20T08:02:33.971 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:33 +0000 (0:00:00.184) 0:02:42.271 ***** 2021-02-20T08:02:34.106 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-02-20T08:02:34.108 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-02-20T08:02:34.149 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:34 +0000 (0:00:00.180) 0:02:42.452 ***** 2021-02-20T08:02:34.176 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-02-20T08:02:34.179 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:34.216 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:34 +0000 (0:00:00.067) 0:02:42.520 ***** 2021-02-20T08:02:34.286 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-02-20T08:02:34.288 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:34.289 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi090.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-02-20T08:02:34.353 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:34 +0000 (0:00:00.136) 0:02:42.656 ***** 2021-02-20T08:02:34.407 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-02-20T08:02:34.409 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:34.419 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:34.430 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:34.441 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:34.452 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:34.491 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:34 +0000 (0:00:00.138) 0:02:42.794 ***** 2021-02-20T08:02:34.521 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T08:02:34.529 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:34.567 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:34 +0000 (0:00:00.076) 0:02:42.871 ***** 2021-02-20T08:02:34.599 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:34.635 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:34 +0000 (0:00:00.068) 0:02:42.939 ***** 2021-02-20T08:02:34.673 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-02-20T08:02:34.675 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:34.715 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:34 +0000 (0:00:00.080) 0:02:43.019 ***** 2021-02-20T08:02:34.742 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:34.782 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:34 +0000 (0:00:00.067) 0:02:43.086 ***** 2021-02-20T08:02:34.810 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-02-20T08:02:34.812 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:34.851 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:34 +0000 (0:00:00.067) 0:02:43.154 ***** 2021-02-20T08:02:34.877 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-02-20T08:02:34.880 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:34.919 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:34 +0000 (0:00:00.068) 0:02:43.222 ***** 2021-02-20T08:02:34.962 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-02-20T08:02:34.964 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:35.015 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:35 +0000 (0:00:00.096) 0:02:43.319 ***** 2021-02-20T08:02:35.053 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-02-20T08:02:35.055 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:35.090 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:35 +0000 (0:00:00.074) 0:02:43.393 ***** 2021-02-20T08:02:35.244 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-02-20T08:02:35.246 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.010964", "end": "2021-02-20 08:02:35.267084", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-02-20 08:02:35.256120", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:35.387 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006004", "end": "2021-02-20 08:02:35.405895", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-02-20 08:02:35.399891", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:35.523 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005526", "end": "2021-02-20 08:02:35.543003", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-02-20 08:02:35.537477", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:35.696 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.019425", "end": "2021-02-20 08:02:35.714637", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-02-20 08:02:35.695212", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:35.762 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:35 +0000 (0:00:00.672) 0:02:44.066 ***** 2021-02-20T08:02:35.801 INFO:teuthology.task.ansible.out: 2021-02-20T08:02:35.803 INFO:teuthology.task.ansible.out:TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-02-20T08:02:35.850 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:35 +0000 (0:00:00.087) 0:02:44.153 ***** 2021-02-20T08:02:35.987 INFO:teuthology.task.ansible.out: 2021-02-20T08:02:35.989 INFO:teuthology.task.ansible.out:TASK [testnode : Touch /ceph-qa-ready] ***************************************** changed: [smithi090.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-02-20T08:02:36.118 INFO:teuthology.task.ansible.out: PLAY [testnodes,senta,vossi,jenkins_slaves] ************************************ 2021-02-20T08:02:36.155 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:36 +0000 (0:00:00.305) 0:02:44.458 ***** 2021-02-20T08:02:36.191 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:36 +0000 (0:00:00.036) 0:02:44.495 ***** 2021-02-20T08:02:36.931 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2021-02-20T08:02:36.933 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] 2021-02-20T08:02:36.987 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] 2021-02-20T08:02:37.032 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:37 +0000 (0:00:00.841) 0:02:45.336 ***** 2021-02-20T08:02:37.072 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-20T08:02:37.074 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-02-20T08:02:37.109 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:37 +0000 (0:00:00.076) 0:02:45.412 ***** 2021-02-20T08:02:37.146 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:37 +0000 (0:00:00.037) 0:02:45.450 ***** 2021-02-20T08:02:37.153 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-20T08:02:37.155 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:37.200 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:37 +0000 (0:00:00.053) 0:02:45.504 ***** 2021-02-20T08:02:37.206 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-20T08:02:37.208 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-02-20T08:02:37.247 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:37 +0000 (0:00:00.047) 0:02:45.551 ***** 2021-02-20T08:02:37.256 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-02-20T08:02:37.258 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-02-20T08:02:37.269 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:37.307 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:37 +0000 (0:00:00.059) 0:02:45.611 ***** 2021-02-20T08:02:37.343 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:37 +0000 (0:00:00.036) 0:02:45.647 ***** 2021-02-20T08:02:37.381 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-02-20T08:02:37.383 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-02-20T08:02:37.428 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:37 +0000 (0:00:00.084) 0:02:45.732 ***** 2021-02-20T08:02:38.221 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-02-20T08:02:38.223 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"cache_update_time": 1613807738, "cache_updated": false, "changed": false} 2021-02-20T08:02:38.290 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:38 +0000 (0:00:00.862) 0:02:46.594 ***** 2021-02-20T08:02:38.311 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"cache_update_time": 1613807781, "cache_updated": false, "changed": false} 2021-02-20T08:02:38.355 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:38 +0000 (0:00:00.064) 0:02:46.658 ***** 2021-02-20T08:02:38.362 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-20T08:02:38.364 INFO:teuthology.task.ansible.out:skipping: [smithi003.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:38.404 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:38 +0000 (0:00:00.049) 0:02:46.708 ***** 2021-02-20T08:02:38.411 INFO:teuthology.task.ansible.out:skipping: [smithi090.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T08:02:38.452 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:38 +0000 (0:00:00.048) 0:02:46.756 ***** 2021-02-20T08:02:38.459 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-20T08:02:38.461 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-02-20T08:02:38.502 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:38 +0000 (0:00:00.049) 0:02:46.805 ***** 2021-02-20T08:02:38.516 INFO:teuthology.task.ansible.out:ok: [smithi090.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-02-20T08:02:38.559 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:38 +0000 (0:00:00.057) 0:02:46.863 ***** 2021-02-20T08:02:38.690 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-02-20T08:02:38.692 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T08:02:38.746 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T08:02:38.801 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T08:02:38.861 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:38 +0000 (0:00:00.301) 0:02:47.165 ***** 2021-02-20T08:02:38.872 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T08:02:38.915 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:38 +0000 (0:00:00.054) 0:02:47.219 ***** 2021-02-20T08:02:39.298 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2021-02-20T08:02:39.300 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613808158.9051604-5049460832560/source", "state": "file", "uid": 0} 2021-02-20T08:02:39.385 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613808158.9648814-127472125146095/source", "state": "file", "uid": 0} 2021-02-20T08:02:39.670 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613808159.3206208-24741558237548/source", "state": "file", "uid": 0} 2021-02-20T08:02:39.716 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:39 +0000 (0:00:00.800) 0:02:48.020 ***** 2021-02-20T08:02:39.724 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613808159.3988206-232364311737826/source", "state": "file", "uid": 0} 2021-02-20T08:02:39.768 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:39 +0000 (0:00:00.052) 0:02:48.072 ***** 2021-02-20T08:02:43.276 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-02-20T08:02:43.277 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-iN2NR8-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-iN2NR8-build", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packag 2021-02-20T08:02:43.279 INFO:teuthology.task.ansible.out:es/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} 2021-02-20T08:02:43.319 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:43 +0000 (0:00:03.550) 0:02:51.622 ***** 2021-02-20T08:02:43.378 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-qNM4Qi-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-qNM4Qi-build", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packag 2021-02-20T08:02:43.380 INFO:teuthology.task.ansible.out:es/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} 2021-02-20T08:02:43.436 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:43 +0000 (0:00:00.117) 0:02:51.740 ***** 2021-02-20T08:02:43.544 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* ok: [smithi090.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} Saturday 20 February 2021 08:02:43 +0000 (0:00:00.067) 0:02:51.807 ***** 2021-02-20T08:02:43.583 INFO:teuthology.task.ansible.out:ok: [smithi003.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-02-20T08:02:43.634 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:43 +0000 (0:00:00.130) 0:02:51.938 ***** 2021-02-20T08:02:43.894 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ changed: [smithi090.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613808163.5451734-216950679292725/source", "state": "file", "uid": 0} 2021-02-20T08:02:43.936 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:43 +0000 (0:00:00.301) 0:02:52.239 ***** 2021-02-20T08:02:43.975 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613808163.6692922-209308366809806/source", "state": "file", "uid": 0} 2021-02-20T08:02:44.020 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:44 +0000 (0:00:00.083) 0:02:52.323 ***** 2021-02-20T08:02:44.198 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2021-02-20T08:02:44.200 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.116106", "end": "2021-02-20 08:02:44.168443", "rc": 0, "start": "2021-02-20 08:02:44.052337", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:44.239 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:44 +0000 (0:00:00.218) 0:02:52.542 ***** 2021-02-20T08:02:44.277 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.119805", "end": "2021-02-20 08:02:44.253406", "rc": 0, "start": "2021-02-20 08:02:44.133601", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T08:02:44.319 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 08:02:44 +0000 (0:00:00.080) 0:02:52.622 ***** 2021-02-20T08:02:47.725 INFO:teuthology.task.ansible.out: TASK [container-host : Restart docker service] ********************************* 2021-02-20T08:02:47.728 INFO:teuthology.task.ansible.out:changed: [smithi003.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "firewalld.service system.slice containerd.service basic.target systemd-journald.socket sysinit.target docker.socket network-online.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "yes", "DelegateControllers": "cpu cpuacct io blkio memory devices pids", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMa 2021-02-20T08:02:47.730 INFO:teuthology.task.ansible.out:inStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128053", "LimitSIGPENDINGSoft": "12805 2021-02-20T08:02:47.734 INFO:teuthology.task.ansible.out:3", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "docker.socket system.slice sysinit.target", "Restart": "always", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 08:01:37 UTC", "StateChangeTimestampMonotonic": "197606387", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "S 2021-02-20T08:02:47.736 INFO:teuthology.task.ansible.out:uccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "containerd.service network-online.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-02-20T08:02:48.042 INFO:teuthology.task.ansible.out:changed: [smithi090.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "docker.socket sysinit.target firewalld.service network-online.target basic.target system.slice containerd.service systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "yes", "DelegateControllers": "cpu cpuacct io blkio memory devices pids", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMa 2021-02-20T08:02:48.044 INFO:teuthology.task.ansible.out:inStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128028", "LimitSIGPENDINGSoft": "12802 2021-02-20T08:02:48.046 INFO:teuthology.task.ansible.out:8", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target docker.socket system.slice", "Restart": "always", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 08:01:42 UTC", "StateChangeTimestampMonotonic": "168465407", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "S 2021-02-20T08:02:48.048 INFO:teuthology.task.ansible.out:uccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "containerd.service network-online.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-02-20T08:02:48.155 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2021-02-20T08:02:48.160 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** 2021-02-20T08:02:48.161 INFO:teuthology.task.ansible.out:skipping: no hosts matched 2021-02-20T08:02:48.167 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched PLAY RECAP ********************************************************************* smithi003.front.sepia.ceph.com : ok=126 changed=38 unreachable=0 failed=0 skipped=184 rescued=0 ignored=0 2021-02-20T08:02:48.169 INFO:teuthology.task.ansible.out:smithi090.front.sepia.ceph.com : ok=126 changed=38 unreachable=0 failed=0 skipped=184 rescued=0 ignored=0 Saturday 20 February 2021 08:02:48 +0000 (0:00:03.848) 0:02:56.471 ***** =============================================================================== users : Create all admin users with sudo access. ----------------------- 22.71s users : Update authorized_keys using the keys repo --------------------- 12.75s container-host : Restart docker service --------------------------------- 3.85s testnode : Ensure packages are not present. ----------------------------- 3.67s container-host : Install registries-conf-ctl ---------------------------- 3.55s testnode : Install packages --------------------------------------------- 3.47s testnode : Zap all non-root disks --------------------------------------- 3.45s testnode : Update apt cache. -------------------------------------------- 2.89s Gathering Facts --------------------------------------------------------- 2.28s users : Clone the keys repo --------------------------------------------- 1.64s testnode : Install apt keys --------------------------------------------- 1.33s ansible-managed : Create the cephlab_sudo sudoers.d file. --------------- 1.11s testnode : Make sure apt dependencies are installed --------------------- 0.99s testnode : Install python-apt ------------------------------------------- 0.98s testnode : Upgrade packages --------------------------------------------- 0.93s testnode : Install packages via pip ------------------------------------- 0.86s container-host : Install container packages ----------------------------- 0.86s Gathering Facts --------------------------------------------------------- 0.84s Gathering Facts --------------------------------------------------------- 0.83s testnode : Create logical volume(s) ------------------------------------- 0.83s 2021-02-20T08:02:48.589 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi003.front.sepia.ceph.com'), Remote(name='ubuntu@smithi090.front.sepia.ceph.com')] 2021-02-20T08:02:48.590 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-20T08:02:48.591 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T08:02:48.716 DEBUG:teuthology.orchestra.run.smithi003:> true 2021-02-20T08:02:48.911 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-20T08:02:48.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T08:02:48.987 DEBUG:teuthology.orchestra.run.smithi090:> true 2021-02-20T08:02:49.183 INFO:teuthology.run_tasks:Running task clock... 2021-02-20T08:02:49.196 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2021-02-20T08:02:49.197 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-02-20T08:02:49.199 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-02-20T08:02:49.260 INFO:teuthology.orchestra.run.smithi003.stdout:20 Feb 08:02:49 ntpd[8285]: ntpd 4.2.8p10@1.3728-o (1): Starting 2021-02-20T08:02:49.260 INFO:teuthology.orchestra.run.smithi003.stdout:20 Feb 08:02:49 ntpd[8285]: Command line: ntpd -gq 2021-02-20T08:02:49.261 INFO:teuthology.orchestra.run.smithi003.stdout:20 Feb 08:02:49 ntpd[8285]: proto: precision = 0.126 usec (-23) 2021-02-20T08:02:49.261 INFO:teuthology.orchestra.run.smithi003.stdout:20 Feb 08:02:49 ntpd[8285]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-02-20T08:02:49.262 INFO:teuthology.orchestra.run.smithi003.stdout:20 Feb 08:02:49 ntpd[8285]: restrict ::: KOD does nothing without LIMITED. 2021-02-20T08:02:49.262 INFO:teuthology.orchestra.run.smithi003.stdout:20 Feb 08:02:49 ntpd[8285]: Listen and drop on 0 v6wildcard [::]:123 2021-02-20T08:02:49.262 INFO:teuthology.orchestra.run.smithi003.stdout:20 Feb 08:02:49 ntpd[8285]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-02-20T08:02:49.262 INFO:teuthology.orchestra.run.smithi003.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-02-20T08:02:49.263 INFO:teuthology.orchestra.run.smithi003.stderr:restrict ::: KOD does nothing without LIMITED. 2021-02-20T08:02:49.264 INFO:teuthology.orchestra.run.smithi003.stdout:20 Feb 08:02:49 ntpd[8285]: Listen normally on 2 lo 127.0.0.1:123 2021-02-20T08:02:49.264 INFO:teuthology.orchestra.run.smithi003.stdout:20 Feb 08:02:49 ntpd[8285]: Listen normally on 3 ens1f0 172.21.15.3:123 2021-02-20T08:02:49.264 INFO:teuthology.orchestra.run.smithi003.stdout:20 Feb 08:02:49 ntpd[8285]: Listen normally on 4 lo [::1]:123 2021-02-20T08:02:49.264 INFO:teuthology.orchestra.run.smithi003.stdout:20 Feb 08:02:49 ntpd[8285]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15dc%3]:123 2021-02-20T08:02:49.264 INFO:teuthology.orchestra.run.smithi003.stdout:20 Feb 08:02:49 ntpd[8285]: Listening on routing socket on fd #22 for interface updates 2021-02-20T08:02:50.261 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:50 ntpd[8285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T08:02:50.261 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:50 ntpd[8285]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T08:02:50.261 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:50 ntpd[8285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T08:02:50.262 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:50 ntpd[8285]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T08:02:52.261 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:52 ntpd[8285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T08:02:52.261 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:52 ntpd[8285]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T08:02:52.261 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:52 ntpd[8285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T08:02:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:52 ntpd[8285]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T08:02:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:52 ntpd[8285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T08:02:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:52 ntpd[8285]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T08:02:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:52 ntpd[8285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T08:02:52.263 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:52 ntpd[8285]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T08:02:53.261 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:53 ntpd[8285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T08:02:53.261 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:53 ntpd[8285]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T08:02:53.261 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:53 ntpd[8285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T08:02:53.261 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:53 ntpd[8285]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T08:02:54.261 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:54 ntpd[8285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T08:02:54.262 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:54 ntpd[8285]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T08:02:54.262 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:54 ntpd[8285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T08:02:54.262 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:54 ntpd[8285]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T08:02:54.262 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:54 ntpd[8285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T08:02:54.263 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:54 ntpd[8285]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T08:02:54.263 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:54 ntpd[8285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T08:02:54.263 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:54 ntpd[8285]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T08:02:54.263 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:54 ntpd[8285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T08:02:54.264 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:54 ntpd[8285]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T08:02:54.264 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:54 ntpd[8285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T08:02:54.264 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:54 ntpd[8285]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T08:02:55.260 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:55 ntpd[8285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T08:02:55.261 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:55 ntpd[8285]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T08:02:55.261 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:55 ntpd[8285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T08:02:55.261 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:55 ntpd[8285]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T08:02:56.261 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:56 ntpd[8285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T08:02:56.262 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:56 ntpd[8285]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T08:02:56.262 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:56 ntpd[8285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T08:02:56.262 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:56 ntpd[8285]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T08:02:56.262 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:56 ntpd[8285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T08:02:56.263 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:56 ntpd[8285]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T08:02:56.263 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:56 ntpd[8285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T08:02:56.263 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:56 ntpd[8285]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T08:02:56.263 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:56 ntpd[8285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T08:02:56.263 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:56 ntpd[8285]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T08:02:56.264 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:56 ntpd[8285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T08:02:56.264 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:56 ntpd[8285]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T08:02:56.264 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:56 ntpd[8285]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-02-20T08:02:56.264 INFO:teuthology.orchestra.run.smithi003.stderr:20 Feb 08:02:56 ntpd[8285]: can't open /var/log/ntpstats/loopstats.20210220: Permission denied 2021-02-20T08:02:56.265 INFO:teuthology.orchestra.run.smithi003.stdout:20 Feb 08:02:56 ntpd[8285]: ntpd: time slew +0.000353 s 2021-02-20T08:02:56.265 INFO:teuthology.orchestra.run.smithi003.stdout:ntpd: time slew +0.000353s 2021-02-20T08:02:56.314 INFO:teuthology.orchestra.run.smithi003.stdout: remote refid st t when poll reach delay offset jitter 2021-02-20T08:02:56.314 INFO:teuthology.orchestra.run.smithi003.stdout:============================================================================== 2021-02-20T08:02:56.315 INFO:teuthology.orchestra.run.smithi003.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T08:02:56.315 INFO:teuthology.orchestra.run.smithi003.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T08:02:56.315 INFO:teuthology.orchestra.run.smithi003.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T08:02:56.315 INFO:teuthology.orchestra.run.smithi003.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T08:02:56.316 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-02-20T08:02:56.316 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-02-20T08:02:56.376 INFO:teuthology.orchestra.run.smithi090.stdout:20 Feb 08:02:56 ntpd[8198]: ntpd 4.2.8p10@1.3728-o (1): Starting 2021-02-20T08:02:56.377 INFO:teuthology.orchestra.run.smithi090.stdout:20 Feb 08:02:56 ntpd[8198]: Command line: ntpd -gq 2021-02-20T08:02:56.379 INFO:teuthology.orchestra.run.smithi090.stdout:20 Feb 08:02:56 ntpd[8198]: proto: precision = 0.123 usec (-23) 2021-02-20T08:02:56.379 INFO:teuthology.orchestra.run.smithi090.stdout:20 Feb 08:02:56 ntpd[8198]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-02-20T08:02:56.379 INFO:teuthology.orchestra.run.smithi090.stdout:20 Feb 08:02:56 ntpd[8198]: restrict ::: KOD does nothing without LIMITED. 2021-02-20T08:02:56.380 INFO:teuthology.orchestra.run.smithi090.stdout:20 Feb 08:02:56 ntpd[8198]: Listen and drop on 0 v6wildcard [::]:123 2021-02-20T08:02:56.380 INFO:teuthology.orchestra.run.smithi090.stdout:20 Feb 08:02:56 ntpd[8198]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-02-20T08:02:56.380 INFO:teuthology.orchestra.run.smithi090.stdout:20 Feb 08:02:56 ntpd[8198]: Listen normally on 2 lo 127.0.0.1:123 2021-02-20T08:02:56.380 INFO:teuthology.orchestra.run.smithi090.stdout:20 Feb 08:02:56 ntpd[8198]: Listen normally on 3 enp3s0f1 172.21.15.90:123 2021-02-20T08:02:56.381 INFO:teuthology.orchestra.run.smithi090.stdout:20 Feb 08:02:56 ntpd[8198]: Listen normally on 4 lo [::1]:123 2021-02-20T08:02:56.381 INFO:teuthology.orchestra.run.smithi090.stdout:20 Feb 08:02:56 ntpd[8198]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d767%5]:123 2021-02-20T08:02:56.381 INFO:teuthology.orchestra.run.smithi090.stdout:20 Feb 08:02:56 ntpd[8198]: Listening on routing socket on fd #22 for interface updates 2021-02-20T08:02:56.381 INFO:teuthology.orchestra.run.smithi090.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-02-20T08:02:56.382 INFO:teuthology.orchestra.run.smithi090.stderr:restrict ::: KOD does nothing without LIMITED. 2021-02-20T08:02:57.378 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:02:57 ntpd[8198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T08:02:57.378 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:02:57 ntpd[8198]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T08:02:57.378 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:02:57 ntpd[8198]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T08:02:57.379 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:02:57 ntpd[8198]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T08:02:57.379 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:02:57 ntpd[8198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T08:02:57.379 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:02:57 ntpd[8198]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T08:02:57.379 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:02:57 ntpd[8198]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T08:02:57.379 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:02:57 ntpd[8198]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T08:02:59.377 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:02:59 ntpd[8198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T08:02:59.378 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:02:59 ntpd[8198]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T08:02:59.378 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:02:59 ntpd[8198]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T08:02:59.378 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:02:59 ntpd[8198]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T08:02:59.379 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:02:59 ntpd[8198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T08:02:59.379 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:02:59 ntpd[8198]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T08:02:59.379 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:02:59 ntpd[8198]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T08:02:59.379 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:02:59 ntpd[8198]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T08:03:00.379 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:00 ntpd[8198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T08:03:00.379 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:00 ntpd[8198]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T08:03:00.380 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:00 ntpd[8198]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T08:03:00.380 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:00 ntpd[8198]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T08:03:01.378 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:01 ntpd[8198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T08:03:01.378 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:01 ntpd[8198]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T08:03:01.378 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:01 ntpd[8198]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T08:03:01.379 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:01 ntpd[8198]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T08:03:01.379 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:01 ntpd[8198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T08:03:01.379 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:01 ntpd[8198]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T08:03:01.379 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:01 ntpd[8198]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T08:03:01.380 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:01 ntpd[8198]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T08:03:01.380 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:01 ntpd[8198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T08:03:01.380 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:01 ntpd[8198]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T08:03:01.380 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:01 ntpd[8198]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T08:03:01.380 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:01 ntpd[8198]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T08:03:02.377 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:02 ntpd[8198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T08:03:02.378 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:02 ntpd[8198]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T08:03:02.378 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:02 ntpd[8198]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T08:03:02.378 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:02 ntpd[8198]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T08:03:03.377 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:03 ntpd[8198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T08:03:03.378 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:03 ntpd[8198]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T08:03:03.378 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:03 ntpd[8198]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T08:03:03.378 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:03 ntpd[8198]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T08:03:03.378 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:03 ntpd[8198]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-02-20T08:03:03.379 INFO:teuthology.orchestra.run.smithi090.stderr:20 Feb 08:03:03 ntpd[8198]: can't open /var/log/ntpstats/loopstats.20210220: Permission denied 2021-02-20T08:03:03.379 INFO:teuthology.orchestra.run.smithi090.stdout:20 Feb 08:03:03 ntpd[8198]: ntpd: time slew +0.000761 s 2021-02-20T08:03:03.379 INFO:teuthology.orchestra.run.smithi090.stdout:ntpd: time slew +0.000761s 2021-02-20T08:03:03.427 INFO:teuthology.orchestra.run.smithi090.stdout: remote refid st t when poll reach delay offset jitter 2021-02-20T08:03:03.427 INFO:teuthology.orchestra.run.smithi090.stdout:============================================================================== 2021-02-20T08:03:03.428 INFO:teuthology.orchestra.run.smithi090.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T08:03:03.428 INFO:teuthology.orchestra.run.smithi090.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T08:03:03.428 INFO:teuthology.orchestra.run.smithi090.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T08:03:03.428 INFO:teuthology.orchestra.run.smithi090.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T08:03:03.429 INFO:teuthology.run_tasks:Running task install... 2021-02-20T08:03:03.439 DEBUG:teuthology.task.install:project ceph 2021-02-20T08:03:03.442 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}} 2021-02-20T08:03:03.443 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-mirror'], 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'} 2021-02-20T08:03:03.444 INFO:teuthology.task.install:Using flavor: basic 2021-02-20T08:03:03.460 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2021-02-20T08:03:03.461 INFO:teuthology.task.install:extra packages: ['rbd-mirror'] 2021-02-20T08:03:03.462 DEBUG:teuthology.orchestra.run.smithi003:> sudo lsb_release -is 2021-02-20T08:03:03.509 INFO:teuthology.orchestra.run.smithi003.stdout:Ubuntu 2021-02-20T08:03:03.514 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-02-20T08:03:03.514 DEBUG:teuthology.orchestra.run.smithi090:> sudo lsb_release -is 2021-02-20T08:03:03.515 DEBUG:teuthology.orchestra.run.smithi003:> sudo apt-key list | grep Ceph 2021-02-20T08:03:03.569 INFO:teuthology.orchestra.run.smithi090.stdout:Ubuntu 2021-02-20T08:03:03.573 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-02-20T08:03:03.574 DEBUG:teuthology.orchestra.run.smithi090:> sudo apt-key list | grep Ceph 2021-02-20T08:03:03.648 INFO:teuthology.orchestra.run.smithi003.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-02-20T08:03:03.669 INFO:teuthology.orchestra.run.smithi090.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-02-20T08:03:03.710 INFO:teuthology.orchestra.run.smithi003.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-02-20T08:03:03.710 INFO:teuthology.orchestra.run.smithi003.stdout:uid [ unknown] Ceph.com (release key) 2021-02-20T08:03:03.711 DEBUG:teuthology.orchestra.run.smithi003:> sudo lsb_release -is 2021-02-20T08:03:03.728 INFO:teuthology.orchestra.run.smithi090.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-02-20T08:03:03.728 INFO:teuthology.orchestra.run.smithi090.stdout:uid [ unknown] Ceph.com (release key) 2021-02-20T08:03:03.729 DEBUG:teuthology.orchestra.run.smithi090:> sudo lsb_release -is 2021-02-20T08:03:03.764 INFO:teuthology.orchestra.run.smithi003.stdout:Ubuntu 2021-02-20T08:03:03.769 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-02-20T08:03:03.770 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-mirror on remote deb x86_64 2021-02-20T08:03:03.770 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=d318d42d502ea13c78f2af2499efd19dcc7e174b 2021-02-20T08:03:03.779 INFO:teuthology.orchestra.run.smithi090.stdout:Ubuntu 2021-02-20T08:03:03.784 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-02-20T08:03:03.784 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-mirror on remote deb x86_64 2021-02-20T08:03:03.785 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=d318d42d502ea13c78f2af2499efd19dcc7e174b 2021-02-20T08:03:03.957 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default/ 2021-02-20T08:03:03.957 INFO:teuthology.task.install.deb:Package version is 17.0.0-929-gd318d42d-1bionic 2021-02-20T08:03:03.963 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default/ 2021-02-20T08:03:03.963 INFO:teuthology.task.install.deb:Package version is 17.0.0-929-gd318d42d-1bionic 2021-02-20T08:03:04.084 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:03:04.085 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-02-20T08:03:04.087 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:03:04.088 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-02-20T08:03:04.103 DEBUG:teuthology.orchestra.run.smithi090:> sudo apt-get update 2021-02-20T08:03:04.104 DEBUG:teuthology.orchestra.run.smithi003:> sudo apt-get update 2021-02-20T08:03:04.291 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-02-20T08:03:04.367 INFO:teuthology.orchestra.run.smithi003.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic InRelease 2021-02-20T08:03:04.371 INFO:teuthology.orchestra.run.smithi090.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic InRelease 2021-02-20T08:03:04.390 INFO:teuthology.orchestra.run.smithi003.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2021-02-20T08:03:04.395 INFO:teuthology.orchestra.run.smithi090.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2021-02-20T08:03:04.404 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-02-20T08:03:04.409 INFO:teuthology.orchestra.run.smithi090.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-02-20T08:03:04.410 INFO:teuthology.orchestra.run.smithi090.stdout:Hit:4 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-02-20T08:03:04.417 INFO:teuthology.orchestra.run.smithi003.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic Release.gpg 2021-02-20T08:03:04.420 INFO:teuthology.orchestra.run.smithi090.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic Release.gpg 2021-02-20T08:03:04.441 INFO:teuthology.orchestra.run.smithi003.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,113 B] 2021-02-20T08:03:04.444 INFO:teuthology.orchestra.run.smithi090.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,113 B] 2021-02-20T08:03:04.479 INFO:teuthology.orchestra.run.smithi090.stdout:Hit:7 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-02-20T08:03:04.481 INFO:teuthology.orchestra.run.smithi003.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 Packages [16.6 kB] 2021-02-20T08:03:04.483 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-02-20T08:03:04.486 INFO:teuthology.orchestra.run.smithi090.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 Packages [16.6 kB] 2021-02-20T08:03:04.557 INFO:teuthology.orchestra.run.smithi090.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-02-20T08:03:04.563 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-02-20T08:03:04.893 INFO:teuthology.orchestra.run.smithi003.stdout:Fetched 27.4 kB in 1s (47.1 kB/s) 2021-02-20T08:03:05.007 INFO:teuthology.orchestra.run.smithi090.stdout:Fetched 27.4 kB in 1s (39.3 kB/s) 2021-02-20T08:03:06.186 INFO:teuthology.orchestra.run.smithi003.stdout:Reading package lists... 2021-02-20T08:03:06.233 DEBUG:teuthology.orchestra.run.smithi003:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.0.0-929-gd318d42d-1bionic cephadm=17.0.0-929-gd318d42d-1bionic ceph-mds=17.0.0-929-gd318d42d-1bionic ceph-mgr=17.0.0-929-gd318d42d-1bionic ceph-common=17.0.0-929-gd318d42d-1bionic ceph-fuse=17.0.0-929-gd318d42d-1bionic ceph-test=17.0.0-929-gd318d42d-1bionic radosgw=17.0.0-929-gd318d42d-1bionic python3-rados=17.0.0-929-gd318d42d-1bionic python3-rgw=17.0.0-929-gd318d42d-1bionic python3-cephfs=17.0.0-929-gd318d42d-1bionic python3-rbd=17.0.0-929-gd318d42d-1bionic libcephfs2=17.0.0-929-gd318d42d-1bionic libcephfs-dev=17.0.0-929-gd318d42d-1bionic librados2=17.0.0-929-gd318d42d-1bionic librbd1=17.0.0-929-gd318d42d-1bionic rbd-fuse=17.0.0-929-gd318d42d-1bionic rbd-mirror=17.0.0-929-gd318d42d-1bionic 2021-02-20T08:03:06.253 INFO:teuthology.orchestra.run.smithi090.stdout:Reading package lists... 2021-02-20T08:03:06.297 DEBUG:teuthology.orchestra.run.smithi090:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.0.0-929-gd318d42d-1bionic cephadm=17.0.0-929-gd318d42d-1bionic ceph-mds=17.0.0-929-gd318d42d-1bionic ceph-mgr=17.0.0-929-gd318d42d-1bionic ceph-common=17.0.0-929-gd318d42d-1bionic ceph-fuse=17.0.0-929-gd318d42d-1bionic ceph-test=17.0.0-929-gd318d42d-1bionic radosgw=17.0.0-929-gd318d42d-1bionic python3-rados=17.0.0-929-gd318d42d-1bionic python3-rgw=17.0.0-929-gd318d42d-1bionic python3-cephfs=17.0.0-929-gd318d42d-1bionic python3-rbd=17.0.0-929-gd318d42d-1bionic libcephfs2=17.0.0-929-gd318d42d-1bionic libcephfs-dev=17.0.0-929-gd318d42d-1bionic librados2=17.0.0-929-gd318d42d-1bionic librbd1=17.0.0-929-gd318d42d-1bionic rbd-fuse=17.0.0-929-gd318d42d-1bionic rbd-mirror=17.0.0-929-gd318d42d-1bionic 2021-02-20T08:03:06.311 INFO:teuthology.orchestra.run.smithi003.stdout:Reading package lists... 2021-02-20T08:03:06.376 INFO:teuthology.orchestra.run.smithi090.stdout:Reading package lists... 2021-02-20T08:03:06.576 INFO:teuthology.orchestra.run.smithi003.stdout:Building dependency tree... 2021-02-20T08:03:06.577 INFO:teuthology.orchestra.run.smithi003.stdout:Reading state information... 2021-02-20T08:03:06.615 INFO:teuthology.orchestra.run.smithi090.stdout:Building dependency tree... 2021-02-20T08:03:06.616 INFO:teuthology.orchestra.run.smithi090.stdout:Reading state information... 2021-02-20T08:03:06.755 INFO:teuthology.orchestra.run.smithi003.stdout:The following additional packages will be installed: 2021-02-20T08:03:06.755 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2021-02-20T08:03:06.756 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd jq 2021-02-20T08:03:06.756 INFO:teuthology.orchestra.run.smithi003.stdout: libjaeger libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2021-02-20T08:03:06.757 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2021-02-20T08:03:06.757 INFO:teuthology.orchestra.run.smithi003.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2021-02-20T08:03:06.757 INFO:teuthology.orchestra.run.smithi003.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-02-20T08:03:06.757 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2021-02-20T08:03:06.757 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy3 python3-cryptography python3-cycler python3-dateutil 2021-02-20T08:03:06.758 INFO:teuthology.orchestra.run.smithi003.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2021-02-20T08:03:06.758 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2021-02-20T08:03:06.758 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-02-20T08:03:06.758 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2021-02-20T08:03:06.758 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2021-02-20T08:03:06.759 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2021-02-20T08:03:06.759 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2021-02-20T08:03:06.759 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2021-02-20T08:03:06.759 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2021-02-20T08:03:06.759 INFO:teuthology.orchestra.run.smithi003.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2021-02-20T08:03:06.760 INFO:teuthology.orchestra.run.smithi003.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2021-02-20T08:03:06.760 INFO:teuthology.orchestra.run.smithi003.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2021-02-20T08:03:06.760 INFO:teuthology.orchestra.run.smithi003.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2021-02-20T08:03:06.760 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2021-02-20T08:03:06.761 INFO:teuthology.orchestra.run.smithi003.stdout:Suggested packages: 2021-02-20T08:03:06.761 INFO:teuthology.orchestra.run.smithi003.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2021-02-20T08:03:06.762 INFO:teuthology.orchestra.run.smithi003.stdout: python-pytest-xdist python-mock python-cryptography-doc 2021-02-20T08:03:06.762 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2021-02-20T08:03:06.762 INFO:teuthology.orchestra.run.smithi003.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2021-02-20T08:03:06.763 INFO:teuthology.orchestra.run.smithi003.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2021-02-20T08:03:06.763 INFO:teuthology.orchestra.run.smithi003.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2021-02-20T08:03:06.763 INFO:teuthology.orchestra.run.smithi003.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2021-02-20T08:03:06.763 INFO:teuthology.orchestra.run.smithi003.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2021-02-20T08:03:06.763 INFO:teuthology.orchestra.run.smithi003.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2021-02-20T08:03:06.764 INFO:teuthology.orchestra.run.smithi003.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2021-02-20T08:03:06.764 INFO:teuthology.orchestra.run.smithi003.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2021-02-20T08:03:06.764 INFO:teuthology.orchestra.run.smithi003.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2021-02-20T08:03:06.764 INFO:teuthology.orchestra.run.smithi003.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2021-02-20T08:03:06.764 INFO:teuthology.orchestra.run.smithi003.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2021-02-20T08:03:06.765 INFO:teuthology.orchestra.run.smithi003.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2021-02-20T08:03:06.781 INFO:teuthology.orchestra.run.smithi090.stdout:The following additional packages will be installed: 2021-02-20T08:03:06.782 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2021-02-20T08:03:06.784 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd jq 2021-02-20T08:03:06.784 INFO:teuthology.orchestra.run.smithi090.stdout: libjaeger libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2021-02-20T08:03:06.784 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2021-02-20T08:03:06.785 INFO:teuthology.orchestra.run.smithi090.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2021-02-20T08:03:06.785 INFO:teuthology.orchestra.run.smithi090.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-02-20T08:03:06.785 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2021-02-20T08:03:06.785 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy3 python3-cryptography python3-cycler python3-dateutil 2021-02-20T08:03:06.785 INFO:teuthology.orchestra.run.smithi090.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2021-02-20T08:03:06.786 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2021-02-20T08:03:06.786 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-02-20T08:03:06.786 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2021-02-20T08:03:06.786 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2021-02-20T08:03:06.786 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2021-02-20T08:03:06.787 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2021-02-20T08:03:06.787 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2021-02-20T08:03:06.787 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2021-02-20T08:03:06.787 INFO:teuthology.orchestra.run.smithi090.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2021-02-20T08:03:06.787 INFO:teuthology.orchestra.run.smithi090.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2021-02-20T08:03:06.788 INFO:teuthology.orchestra.run.smithi090.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2021-02-20T08:03:06.788 INFO:teuthology.orchestra.run.smithi090.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2021-02-20T08:03:06.788 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2021-02-20T08:03:06.789 INFO:teuthology.orchestra.run.smithi090.stdout:Suggested packages: 2021-02-20T08:03:06.789 INFO:teuthology.orchestra.run.smithi090.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2021-02-20T08:03:06.790 INFO:teuthology.orchestra.run.smithi090.stdout: python-pytest-xdist python-mock python-cryptography-doc 2021-02-20T08:03:06.791 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2021-02-20T08:03:06.791 INFO:teuthology.orchestra.run.smithi090.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2021-02-20T08:03:06.791 INFO:teuthology.orchestra.run.smithi090.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2021-02-20T08:03:06.791 INFO:teuthology.orchestra.run.smithi090.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2021-02-20T08:03:06.792 INFO:teuthology.orchestra.run.smithi090.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2021-02-20T08:03:06.792 INFO:teuthology.orchestra.run.smithi090.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2021-02-20T08:03:06.792 INFO:teuthology.orchestra.run.smithi090.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2021-02-20T08:03:06.792 INFO:teuthology.orchestra.run.smithi090.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2021-02-20T08:03:06.792 INFO:teuthology.orchestra.run.smithi090.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2021-02-20T08:03:06.793 INFO:teuthology.orchestra.run.smithi090.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2021-02-20T08:03:06.793 INFO:teuthology.orchestra.run.smithi090.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2021-02-20T08:03:06.793 INFO:teuthology.orchestra.run.smithi090.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2021-02-20T08:03:06.793 INFO:teuthology.orchestra.run.smithi090.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2021-02-20T08:03:06.870 INFO:teuthology.orchestra.run.smithi003.stdout:The following NEW packages will be installed: 2021-02-20T08:03:06.871 INFO:teuthology.orchestra.run.smithi003.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-02-20T08:03:06.871 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2021-02-20T08:03:06.871 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm jq 2021-02-20T08:03:06.872 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-dev libcephfs2 libjaeger libjq1 libleveldb1v5 liblttng-ust0 2021-02-20T08:03:06.873 INFO:teuthology.orchestra.run.smithi003.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2021-02-20T08:03:06.873 INFO:teuthology.orchestra.run.smithi003.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2021-02-20T08:03:06.873 INFO:teuthology.orchestra.run.smithi003.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2021-02-20T08:03:06.873 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-02-20T08:03:06.874 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2021-02-20T08:03:06.874 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi-backend python3-cherrypy3 python3-cryptography python3-cycler 2021-02-20T08:03:06.874 INFO:teuthology.orchestra.run.smithi003.stdout: python3-dateutil python3-decorator python3-fasteners python3-html5lib 2021-02-20T08:03:06.875 INFO:teuthology.orchestra.run.smithi003.stdout: python3-httplib2 python3-jinja2 python3-joblib python3-json-pointer 2021-02-20T08:03:06.875 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch python3-jwt python3-kubernetes python3-logutils 2021-02-20T08:03:06.876 INFO:teuthology.orchestra.run.smithi003.stdout: python3-lxml python3-mako python3-markupsafe python3-matplotlib 2021-02-20T08:03:06.876 INFO:teuthology.orchestra.run.smithi003.stdout: python3-monotonic python3-numpy python3-oauth2client python3-olefile 2021-02-20T08:03:06.876 INFO:teuthology.orchestra.run.smithi003.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2021-02-20T08:03:06.877 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan python3-pil python3-pluggy python3-prettytable python3-py 2021-02-20T08:03:06.877 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyparsing 2021-02-20T08:03:06.877 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2021-02-20T08:03:06.877 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2021-02-20T08:03:06.878 INFO:teuthology.orchestra.run.smithi003.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2021-02-20T08:03:06.878 INFO:teuthology.orchestra.run.smithi003.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2021-02-20T08:03:06.879 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2021-02-20T08:03:06.879 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2021-02-20T08:03:06.880 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug radosgw rbd-fuse rbd-mirror socat xmlstarlet 2021-02-20T08:03:06.881 INFO:teuthology.orchestra.run.smithi003.stdout:The following packages will be upgraded: 2021-02-20T08:03:06.881 INFO:teuthology.orchestra.run.smithi003.stdout: librados2 librbd1 2021-02-20T08:03:06.892 INFO:teuthology.orchestra.run.smithi090.stdout:The following NEW packages will be installed: 2021-02-20T08:03:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-02-20T08:03:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2021-02-20T08:03:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm jq 2021-02-20T08:03:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-dev libcephfs2 libjaeger libjq1 libleveldb1v5 liblttng-ust0 2021-02-20T08:03:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2021-02-20T08:03:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2021-02-20T08:03:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2021-02-20T08:03:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-02-20T08:03:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2021-02-20T08:03:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi-backend python3-cherrypy3 python3-cryptography python3-cycler 2021-02-20T08:03:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: python3-dateutil python3-decorator python3-fasteners python3-html5lib 2021-02-20T08:03:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: python3-httplib2 python3-jinja2 python3-joblib python3-json-pointer 2021-02-20T08:03:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch python3-jwt python3-kubernetes python3-logutils 2021-02-20T08:03:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: python3-lxml python3-mako python3-markupsafe python3-matplotlib 2021-02-20T08:03:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: python3-monotonic python3-numpy python3-oauth2client python3-olefile 2021-02-20T08:03:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2021-02-20T08:03:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan python3-pil python3-pluggy python3-prettytable python3-py 2021-02-20T08:03:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyparsing 2021-02-20T08:03:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2021-02-20T08:03:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2021-02-20T08:03:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2021-02-20T08:03:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2021-02-20T08:03:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2021-02-20T08:03:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2021-02-20T08:03:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug radosgw rbd-fuse rbd-mirror socat xmlstarlet 2021-02-20T08:03:06.898 INFO:teuthology.orchestra.run.smithi090.stdout:The following packages will be upgraded: 2021-02-20T08:03:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: librados2 librbd1 2021-02-20T08:03:07.016 INFO:teuthology.orchestra.run.smithi003.stdout:2 upgraded, 112 newly installed, 0 to remove and 34 not upgraded. 2021-02-20T08:03:07.016 INFO:teuthology.orchestra.run.smithi003.stdout:Need to get 169 MB of archives. 2021-02-20T08:03:07.016 INFO:teuthology.orchestra.run.smithi003.stdout:After this operation, 793 MB of additional disk space will be used. 2021-02-20T08:03:07.017 INFO:teuthology.orchestra.run.smithi003.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 libjaeger amd64 17.0.0-929-gd318d42d-1bionic [3,712 B] 2021-02-20T08:03:07.034 INFO:teuthology.orchestra.run.smithi090.stdout:2 upgraded, 112 newly installed, 0 to remove and 34 not upgraded. 2021-02-20T08:03:07.035 INFO:teuthology.orchestra.run.smithi090.stdout:Need to get 169 MB of archives. 2021-02-20T08:03:07.035 INFO:teuthology.orchestra.run.smithi090.stdout:After this operation, 793 MB of additional disk space will be used. 2021-02-20T08:03:07.035 INFO:teuthology.orchestra.run.smithi090.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 libjaeger amd64 17.0.0-929-gd318d42d-1bionic [3,712 B] 2021-02-20T08:03:07.062 INFO:teuthology.orchestra.run.smithi003.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 17.0.0-929-gd318d42d-1bionic [3,122 kB] 2021-02-20T08:03:07.064 INFO:teuthology.orchestra.run.smithi003.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2021-02-20T08:03:07.082 INFO:teuthology.orchestra.run.smithi090.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2021-02-20T08:03:07.082 INFO:teuthology.orchestra.run.smithi090.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 17.0.0-929-gd318d42d-1bionic [3,122 kB] 2021-02-20T08:03:07.144 INFO:teuthology.orchestra.run.smithi003.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2021-02-20T08:03:07.162 INFO:teuthology.orchestra.run.smithi090.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2021-02-20T08:03:07.330 INFO:teuthology.orchestra.run.smithi003.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 17.0.0-929-gd318d42d-1bionic [3,162 kB] 2021-02-20T08:03:07.356 INFO:teuthology.orchestra.run.smithi090.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 17.0.0-929-gd318d42d-1bionic [3,162 kB] 2021-02-20T08:03:07.394 INFO:teuthology.orchestra.run.smithi003.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2021-02-20T08:03:07.399 INFO:teuthology.orchestra.run.smithi003.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2021-02-20T08:03:07.414 INFO:teuthology.orchestra.run.smithi003.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2021-02-20T08:03:07.418 INFO:teuthology.orchestra.run.smithi090.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2021-02-20T08:03:07.437 INFO:teuthology.orchestra.run.smithi090.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2021-02-20T08:03:07.456 INFO:teuthology.orchestra.run.smithi090.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2021-02-20T08:03:07.513 INFO:teuthology.orchestra.run.smithi003.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2021-02-20T08:03:07.517 INFO:teuthology.orchestra.run.smithi003.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2021-02-20T08:03:07.533 INFO:teuthology.orchestra.run.smithi003.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 17.0.0-929-gd318d42d-1bionic [664 kB] 2021-02-20T08:03:07.549 INFO:teuthology.orchestra.run.smithi003.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2021-02-20T08:03:07.557 INFO:teuthology.orchestra.run.smithi003.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2021-02-20T08:03:07.558 INFO:teuthology.orchestra.run.smithi090.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2021-02-20T08:03:07.562 INFO:teuthology.orchestra.run.smithi090.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2021-02-20T08:03:07.565 INFO:teuthology.orchestra.run.smithi003.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2021-02-20T08:03:07.566 INFO:teuthology.orchestra.run.smithi090.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 17.0.0-929-gd318d42d-1bionic [664 kB] 2021-02-20T08:03:07.576 INFO:teuthology.orchestra.run.smithi003.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2021-02-20T08:03:07.576 INFO:teuthology.orchestra.run.smithi003.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 17.0.0-929-gd318d42d-1bionic [339 kB] 2021-02-20T08:03:07.584 INFO:teuthology.orchestra.run.smithi003.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2021-02-20T08:03:07.588 INFO:teuthology.orchestra.run.smithi090.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2021-02-20T08:03:07.595 INFO:teuthology.orchestra.run.smithi090.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2021-02-20T08:03:07.601 INFO:teuthology.orchestra.run.smithi003.stdout:Get:18 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 17.0.0-929-gd318d42d-1bionic [21.7 kB] 2021-02-20T08:03:07.602 INFO:teuthology.orchestra.run.smithi003.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 17.0.0-929-gd318d42d-1bionic [162 kB] 2021-02-20T08:03:07.604 INFO:teuthology.orchestra.run.smithi090.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2021-02-20T08:03:07.607 INFO:teuthology.orchestra.run.smithi003.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2021-02-20T08:03:07.610 INFO:teuthology.orchestra.run.smithi003.stdout:Get:21 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 17.0.0-929-gd318d42d-1bionic [29.2 kB] 2021-02-20T08:03:07.611 INFO:teuthology.orchestra.run.smithi090.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 17.0.0-929-gd318d42d-1bionic [339 kB] 2021-02-20T08:03:07.611 INFO:teuthology.orchestra.run.smithi003.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2021-02-20T08:03:07.612 INFO:teuthology.orchestra.run.smithi003.stdout:Get:23 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 17.0.0-929-gd318d42d-1bionic [336 kB] 2021-02-20T08:03:07.614 INFO:teuthology.orchestra.run.smithi090.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2021-02-20T08:03:07.623 INFO:teuthology.orchestra.run.smithi090.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2021-02-20T08:03:07.627 INFO:teuthology.orchestra.run.smithi003.stdout:Get:24 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 17.0.0-929-gd318d42d-1bionic [3,295 kB] 2021-02-20T08:03:07.637 INFO:teuthology.orchestra.run.smithi090.stdout:Get:18 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 17.0.0-929-gd318d42d-1bionic [21.7 kB] 2021-02-20T08:03:07.638 INFO:teuthology.orchestra.run.smithi090.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 17.0.0-929-gd318d42d-1bionic [162 kB] 2021-02-20T08:03:07.638 INFO:teuthology.orchestra.run.smithi003.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2021-02-20T08:03:07.649 INFO:teuthology.orchestra.run.smithi003.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2021-02-20T08:03:07.649 INFO:teuthology.orchestra.run.smithi003.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2021-02-20T08:03:07.649 INFO:teuthology.orchestra.run.smithi090.stdout:Get:20 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 17.0.0-929-gd318d42d-1bionic [29.2 kB] 2021-02-20T08:03:07.650 INFO:teuthology.orchestra.run.smithi090.stdout:Get:21 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 17.0.0-929-gd318d42d-1bionic [336 kB] 2021-02-20T08:03:07.658 INFO:teuthology.orchestra.run.smithi090.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2021-02-20T08:03:07.659 INFO:teuthology.orchestra.run.smithi090.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2021-02-20T08:03:07.662 INFO:teuthology.orchestra.run.smithi090.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2021-02-20T08:03:07.667 INFO:teuthology.orchestra.run.smithi090.stdout:Get:25 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 17.0.0-929-gd318d42d-1bionic [3,295 kB] 2021-02-20T08:03:07.675 INFO:teuthology.orchestra.run.smithi090.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2021-02-20T08:03:07.712 INFO:teuthology.orchestra.run.smithi003.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2021-02-20T08:03:07.712 INFO:teuthology.orchestra.run.smithi003.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2021-02-20T08:03:07.715 INFO:teuthology.orchestra.run.smithi003.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2021-02-20T08:03:07.716 INFO:teuthology.orchestra.run.smithi003.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2021-02-20T08:03:07.716 INFO:teuthology.orchestra.run.smithi003.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2021-02-20T08:03:07.719 INFO:teuthology.orchestra.run.smithi003.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2021-02-20T08:03:07.721 INFO:teuthology.orchestra.run.smithi003.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2021-02-20T08:03:07.741 INFO:teuthology.orchestra.run.smithi090.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2021-02-20T08:03:07.766 INFO:teuthology.orchestra.run.smithi090.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2021-02-20T08:03:07.766 INFO:teuthology.orchestra.run.smithi090.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2021-02-20T08:03:07.768 INFO:teuthology.orchestra.run.smithi090.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2021-02-20T08:03:07.769 INFO:teuthology.orchestra.run.smithi090.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2021-02-20T08:03:07.770 INFO:teuthology.orchestra.run.smithi090.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2021-02-20T08:03:07.772 INFO:teuthology.orchestra.run.smithi090.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2021-02-20T08:03:07.775 INFO:teuthology.orchestra.run.smithi090.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2021-02-20T08:03:07.791 INFO:teuthology.orchestra.run.smithi003.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2021-02-20T08:03:07.792 INFO:teuthology.orchestra.run.smithi003.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2021-02-20T08:03:07.813 INFO:teuthology.orchestra.run.smithi003.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2021-02-20T08:03:07.820 INFO:teuthology.orchestra.run.smithi090.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2021-02-20T08:03:07.821 INFO:teuthology.orchestra.run.smithi090.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2021-02-20T08:03:07.844 INFO:teuthology.orchestra.run.smithi003.stdout:Get:38 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 17.0.0-929-gd318d42d-1bionic [99.2 kB] 2021-02-20T08:03:07.847 INFO:teuthology.orchestra.run.smithi003.stdout:Get:39 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 17.0.0-929-gd318d42d-1bionic [414 kB] 2021-02-20T08:03:07.870 INFO:teuthology.orchestra.run.smithi003.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2021-02-20T08:03:07.871 INFO:teuthology.orchestra.run.smithi003.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2021-02-20T08:03:07.873 INFO:teuthology.orchestra.run.smithi003.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2021-02-20T08:03:07.875 INFO:teuthology.orchestra.run.smithi003.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2021-02-20T08:03:07.879 INFO:teuthology.orchestra.run.smithi003.stdout:Get:44 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 17.0.0-929-gd318d42d-1bionic [21.2 MB] 2021-02-20T08:03:07.883 INFO:teuthology.orchestra.run.smithi003.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2021-02-20T08:03:07.888 INFO:teuthology.orchestra.run.smithi003.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2021-02-20T08:03:07.888 INFO:teuthology.orchestra.run.smithi003.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2021-02-20T08:03:07.891 INFO:teuthology.orchestra.run.smithi090.stdout:Get:37 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 17.0.0-929-gd318d42d-1bionic [99.2 kB] 2021-02-20T08:03:07.895 INFO:teuthology.orchestra.run.smithi090.stdout:Get:38 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 17.0.0-929-gd318d42d-1bionic [414 kB] 2021-02-20T08:03:07.899 INFO:teuthology.orchestra.run.smithi090.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2021-02-20T08:03:07.900 INFO:teuthology.orchestra.run.smithi090.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2021-02-20T08:03:07.901 INFO:teuthology.orchestra.run.smithi090.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2021-02-20T08:03:07.902 INFO:teuthology.orchestra.run.smithi090.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2021-02-20T08:03:07.905 INFO:teuthology.orchestra.run.smithi090.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2021-02-20T08:03:07.913 INFO:teuthology.orchestra.run.smithi090.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2021-02-20T08:03:07.918 INFO:teuthology.orchestra.run.smithi090.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2021-02-20T08:03:07.919 INFO:teuthology.orchestra.run.smithi090.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2021-02-20T08:03:07.922 INFO:teuthology.orchestra.run.smithi090.stdout:Get:47 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 17.0.0-929-gd318d42d-1bionic [21.2 MB] 2021-02-20T08:03:07.949 INFO:teuthology.orchestra.run.smithi003.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2021-02-20T08:03:07.978 INFO:teuthology.orchestra.run.smithi090.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2021-02-20T08:03:08.038 INFO:teuthology.orchestra.run.smithi003.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2021-02-20T08:03:08.039 INFO:teuthology.orchestra.run.smithi003.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2021-02-20T08:03:08.061 INFO:teuthology.orchestra.run.smithi090.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2021-02-20T08:03:08.061 INFO:teuthology.orchestra.run.smithi090.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2021-02-20T08:03:08.315 INFO:teuthology.orchestra.run.smithi003.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2021-02-20T08:03:08.348 INFO:teuthology.orchestra.run.smithi090.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2021-02-20T08:03:08.357 INFO:teuthology.orchestra.run.smithi003.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2021-02-20T08:03:08.358 INFO:teuthology.orchestra.run.smithi003.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2021-02-20T08:03:08.380 INFO:teuthology.orchestra.run.smithi090.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2021-02-20T08:03:08.382 INFO:teuthology.orchestra.run.smithi090.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2021-02-20T08:03:08.393 INFO:teuthology.orchestra.run.smithi003.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2021-02-20T08:03:08.394 INFO:teuthology.orchestra.run.smithi003.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2021-02-20T08:03:08.394 INFO:teuthology.orchestra.run.smithi003.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2021-02-20T08:03:08.394 INFO:teuthology.orchestra.run.smithi003.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2021-02-20T08:03:08.395 INFO:teuthology.orchestra.run.smithi003.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2021-02-20T08:03:08.395 INFO:teuthology.orchestra.run.smithi003.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2021-02-20T08:03:08.395 INFO:teuthology.orchestra.run.smithi003.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2021-02-20T08:03:08.422 INFO:teuthology.orchestra.run.smithi090.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2021-02-20T08:03:08.423 INFO:teuthology.orchestra.run.smithi090.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2021-02-20T08:03:08.424 INFO:teuthology.orchestra.run.smithi090.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2021-02-20T08:03:08.424 INFO:teuthology.orchestra.run.smithi090.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2021-02-20T08:03:08.425 INFO:teuthology.orchestra.run.smithi090.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2021-02-20T08:03:08.425 INFO:teuthology.orchestra.run.smithi090.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2021-02-20T08:03:08.428 INFO:teuthology.orchestra.run.smithi090.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2021-02-20T08:03:08.474 INFO:teuthology.orchestra.run.smithi003.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2021-02-20T08:03:08.475 INFO:teuthology.orchestra.run.smithi003.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2021-02-20T08:03:08.480 INFO:teuthology.orchestra.run.smithi003.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2021-02-20T08:03:08.481 INFO:teuthology.orchestra.run.smithi003.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2021-02-20T08:03:08.486 INFO:teuthology.orchestra.run.smithi003.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2021-02-20T08:03:08.486 INFO:teuthology.orchestra.run.smithi003.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2021-02-20T08:03:08.506 INFO:teuthology.orchestra.run.smithi090.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2021-02-20T08:03:08.508 INFO:teuthology.orchestra.run.smithi090.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2021-02-20T08:03:08.515 INFO:teuthology.orchestra.run.smithi090.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2021-02-20T08:03:08.516 INFO:teuthology.orchestra.run.smithi090.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2021-02-20T08:03:08.521 INFO:teuthology.orchestra.run.smithi090.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2021-02-20T08:03:08.522 INFO:teuthology.orchestra.run.smithi090.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2021-02-20T08:03:08.522 INFO:teuthology.orchestra.run.smithi090.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2021-02-20T08:03:08.523 INFO:teuthology.orchestra.run.smithi090.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2021-02-20T08:03:08.524 INFO:teuthology.orchestra.run.smithi090.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2021-02-20T08:03:08.554 INFO:teuthology.orchestra.run.smithi003.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2021-02-20T08:03:08.559 INFO:teuthology.orchestra.run.smithi003.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2021-02-20T08:03:08.562 INFO:teuthology.orchestra.run.smithi003.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2021-02-20T08:03:08.563 INFO:teuthology.orchestra.run.smithi003.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2021-02-20T08:03:08.588 INFO:teuthology.orchestra.run.smithi090.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2021-02-20T08:03:08.605 INFO:teuthology.orchestra.run.smithi090.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2021-02-20T08:03:08.608 INFO:teuthology.orchestra.run.smithi090.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2021-02-20T08:03:08.615 INFO:teuthology.orchestra.run.smithi090.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2021-02-20T08:03:08.618 INFO:teuthology.orchestra.run.smithi090.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2021-02-20T08:03:08.619 INFO:teuthology.orchestra.run.smithi090.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2021-02-20T08:03:08.619 INFO:teuthology.orchestra.run.smithi090.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2021-02-20T08:03:08.620 INFO:teuthology.orchestra.run.smithi090.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2021-02-20T08:03:08.620 INFO:teuthology.orchestra.run.smithi090.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2021-02-20T08:03:08.633 INFO:teuthology.orchestra.run.smithi003.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2021-02-20T08:03:08.639 INFO:teuthology.orchestra.run.smithi003.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2021-02-20T08:03:08.647 INFO:teuthology.orchestra.run.smithi003.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2021-02-20T08:03:08.650 INFO:teuthology.orchestra.run.smithi003.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2021-02-20T08:03:08.650 INFO:teuthology.orchestra.run.smithi003.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2021-02-20T08:03:08.651 INFO:teuthology.orchestra.run.smithi003.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2021-02-20T08:03:08.667 INFO:teuthology.orchestra.run.smithi090.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2021-02-20T08:03:08.701 INFO:teuthology.orchestra.run.smithi090.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2021-02-20T08:03:08.702 INFO:teuthology.orchestra.run.smithi090.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2021-02-20T08:03:08.702 INFO:teuthology.orchestra.run.smithi090.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2021-02-20T08:03:08.706 INFO:teuthology.orchestra.run.smithi090.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.3 [1,097 kB] 2021-02-20T08:03:08.712 INFO:teuthology.orchestra.run.smithi003.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2021-02-20T08:03:08.714 INFO:teuthology.orchestra.run.smithi003.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2021-02-20T08:03:08.719 INFO:teuthology.orchestra.run.smithi003.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2021-02-20T08:03:08.720 INFO:teuthology.orchestra.run.smithi003.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2021-02-20T08:03:08.726 INFO:teuthology.orchestra.run.smithi090.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2021-02-20T08:03:08.727 INFO:teuthology.orchestra.run.smithi090.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2021-02-20T08:03:08.727 INFO:teuthology.orchestra.run.smithi090.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2021-02-20T08:03:08.792 INFO:teuthology.orchestra.run.smithi003.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2021-02-20T08:03:08.793 INFO:teuthology.orchestra.run.smithi003.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2021-02-20T08:03:08.795 INFO:teuthology.orchestra.run.smithi003.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.3 [1,097 kB] 2021-02-20T08:03:08.815 INFO:teuthology.orchestra.run.smithi003.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2021-02-20T08:03:08.816 INFO:teuthology.orchestra.run.smithi003.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2021-02-20T08:03:08.816 INFO:teuthology.orchestra.run.smithi003.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2021-02-20T08:03:08.833 INFO:teuthology.orchestra.run.smithi090.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2021-02-20T08:03:08.834 INFO:teuthology.orchestra.run.smithi090.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2021-02-20T08:03:08.835 INFO:teuthology.orchestra.run.smithi090.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.4 [329 kB] 2021-02-20T08:03:08.840 INFO:teuthology.orchestra.run.smithi090.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2021-02-20T08:03:08.841 INFO:teuthology.orchestra.run.smithi090.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2021-02-20T08:03:08.841 INFO:teuthology.orchestra.run.smithi090.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2021-02-20T08:03:08.842 INFO:teuthology.orchestra.run.smithi090.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2021-02-20T08:03:08.842 INFO:teuthology.orchestra.run.smithi090.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2021-02-20T08:03:08.843 INFO:teuthology.orchestra.run.smithi090.stdout:Get:95 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2021-02-20T08:03:08.956 INFO:teuthology.orchestra.run.smithi003.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2021-02-20T08:03:08.956 INFO:teuthology.orchestra.run.smithi003.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2021-02-20T08:03:08.958 INFO:teuthology.orchestra.run.smithi003.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.4 [329 kB] 2021-02-20T08:03:08.966 INFO:teuthology.orchestra.run.smithi003.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2021-02-20T08:03:08.966 INFO:teuthology.orchestra.run.smithi003.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2021-02-20T08:03:08.967 INFO:teuthology.orchestra.run.smithi003.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2021-02-20T08:03:08.967 INFO:teuthology.orchestra.run.smithi003.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2021-02-20T08:03:08.968 INFO:teuthology.orchestra.run.smithi003.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2021-02-20T08:03:08.969 INFO:teuthology.orchestra.run.smithi003.stdout:Get:95 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2021-02-20T08:03:09.263 INFO:teuthology.orchestra.run.smithi003.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 17.0.0-929-gd318d42d-1bionic [5,630 kB] 2021-02-20T08:03:09.330 INFO:teuthology.orchestra.run.smithi090.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 17.0.0-929-gd318d42d-1bionic [5,630 kB] 2021-02-20T08:03:09.632 INFO:teuthology.orchestra.run.smithi003.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 17.0.0-929-gd318d42d-1bionic [185 kB] 2021-02-20T08:03:09.639 INFO:teuthology.orchestra.run.smithi003.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 17.0.0-929-gd318d42d-1bionic [1,366 kB] 2021-02-20T08:03:09.687 INFO:teuthology.orchestra.run.smithi090.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 17.0.0-929-gd318d42d-1bionic [185 kB] 2021-02-20T08:03:09.698 INFO:teuthology.orchestra.run.smithi090.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 17.0.0-929-gd318d42d-1bionic [1,366 kB] 2021-02-20T08:03:09.728 INFO:teuthology.orchestra.run.smithi003.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 17.0.0-929-gd318d42d-1bionic [6,660 kB] 2021-02-20T08:03:09.795 INFO:teuthology.orchestra.run.smithi090.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 17.0.0-929-gd318d42d-1bionic [6,660 kB] 2021-02-20T08:03:10.160 INFO:teuthology.orchestra.run.smithi003.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 17.0.0-929-gd318d42d-1bionic [24.9 MB] 2021-02-20T08:03:10.235 INFO:teuthology.orchestra.run.smithi090.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 17.0.0-929-gd318d42d-1bionic [24.9 MB] 2021-02-20T08:03:11.782 INFO:teuthology.orchestra.run.smithi003.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 17.0.0-929-gd318d42d-1bionic [3,804 B] 2021-02-20T08:03:11.783 INFO:teuthology.orchestra.run.smithi003.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 17.0.0-929-gd318d42d-1bionic [770 kB] 2021-02-20T08:03:11.829 INFO:teuthology.orchestra.run.smithi003.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 17.0.0-929-gd318d42d-1bionic [2,154 kB] 2021-02-20T08:03:11.878 INFO:teuthology.orchestra.run.smithi090.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 17.0.0-929-gd318d42d-1bionic [3,804 B] 2021-02-20T08:03:11.879 INFO:teuthology.orchestra.run.smithi090.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 17.0.0-929-gd318d42d-1bionic [770 kB] 2021-02-20T08:03:11.930 INFO:teuthology.orchestra.run.smithi090.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 17.0.0-929-gd318d42d-1bionic [2,154 kB] 2021-02-20T08:03:11.976 INFO:teuthology.orchestra.run.smithi003.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 17.0.0-929-gd318d42d-1bionic [62.2 kB] 2021-02-20T08:03:11.978 INFO:teuthology.orchestra.run.smithi003.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 17.0.0-929-gd318d42d-1bionic [67.9 kB] 2021-02-20T08:03:11.980 INFO:teuthology.orchestra.run.smithi003.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 17.0.0-929-gd318d42d-1bionic [2,328 kB] 2021-02-20T08:03:12.074 INFO:teuthology.orchestra.run.smithi090.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 17.0.0-929-gd318d42d-1bionic [62.2 kB] 2021-02-20T08:03:12.080 INFO:teuthology.orchestra.run.smithi090.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 17.0.0-929-gd318d42d-1bionic [67.9 kB] 2021-02-20T08:03:12.082 INFO:teuthology.orchestra.run.smithi090.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 17.0.0-929-gd318d42d-1bionic [2,328 kB] 2021-02-20T08:03:12.137 INFO:teuthology.orchestra.run.smithi003.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 17.0.0-929-gd318d42d-1bionic [7,884 kB] 2021-02-20T08:03:12.236 INFO:teuthology.orchestra.run.smithi090.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 17.0.0-929-gd318d42d-1bionic [7,884 kB] 2021-02-20T08:03:12.676 INFO:teuthology.orchestra.run.smithi003.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 17.0.0-929-gd318d42d-1bionic [3,924 B] 2021-02-20T08:03:12.677 INFO:teuthology.orchestra.run.smithi003.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 17.0.0-929-gd318d42d-1bionic [23.3 kB] 2021-02-20T08:03:12.678 INFO:teuthology.orchestra.run.smithi003.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 17.0.0-929-gd318d42d-1bionic [45.0 MB] 2021-02-20T08:03:12.723 INFO:teuthology.orchestra.run.smithi090.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 17.0.0-929-gd318d42d-1bionic [3,924 B] 2021-02-20T08:03:12.723 INFO:teuthology.orchestra.run.smithi090.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 17.0.0-929-gd318d42d-1bionic [23.3 kB] 2021-02-20T08:03:12.724 INFO:teuthology.orchestra.run.smithi090.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 17.0.0-929-gd318d42d-1bionic [45.0 MB] 2021-02-20T08:03:15.317 INFO:teuthology.orchestra.run.smithi090.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 17.0.0-929-gd318d42d-1bionic [19.8 kB] 2021-02-20T08:03:15.318 INFO:teuthology.orchestra.run.smithi090.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 17.0.0-929-gd318d42d-1bionic [10.1 MB] 2021-02-20T08:03:15.914 INFO:teuthology.orchestra.run.smithi090.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 17.0.0-929-gd318d42d-1bionic [77.6 kB] 2021-02-20T08:03:15.916 INFO:teuthology.orchestra.run.smithi090.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 rbd-mirror amd64 17.0.0-929-gd318d42d-1bionic [3,331 kB] 2021-02-20T08:03:16.079 INFO:teuthology.orchestra.run.smithi003.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 17.0.0-929-gd318d42d-1bionic [19.8 kB] 2021-02-20T08:03:16.080 INFO:teuthology.orchestra.run.smithi003.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 17.0.0-929-gd318d42d-1bionic [10.1 MB] 2021-02-20T08:03:16.429 INFO:teuthology.orchestra.run.smithi003.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 17.0.0-929-gd318d42d-1bionic [77.6 kB] 2021-02-20T08:03:16.431 INFO:teuthology.orchestra.run.smithi003.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 rbd-mirror amd64 17.0.0-929-gd318d42d-1bionic [3,331 kB] 2021-02-20T08:03:16.826 INFO:teuthology.orchestra.run.smithi090.stdout:Fetched 169 MB in 9s (18.4 MB/s) 2021-02-20T08:03:16.971 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-uritemplate. 2021-02-20T08:03:17.021 INFO:teuthology.orchestra.run.smithi090.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137611 files and directories currently installed.) 2021-02-20T08:03:17.025 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../000-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2021-02-20T08:03:17.062 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2021-02-20T08:03:17.260 INFO:teuthology.orchestra.run.smithi003.stdout:Fetched 169 MB in 10s (17.6 MB/s) 2021-02-20T08:03:17.307 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libjaeger. 2021-02-20T08:03:17.321 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../001-libjaeger_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:17.366 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libjaeger (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:17.381 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-uritemplate. 2021-02-20T08:03:17.430 INFO:teuthology.orchestra.run.smithi003.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137611 files and directories currently installed.) 2021-02-20T08:03:17.434 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../000-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2021-02-20T08:03:17.471 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2021-02-20T08:03:17.702 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-02-20T08:03:17.715 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../002-liblttng-ust0_2.10.1-1_amd64.deb ... 2021-02-20T08:03:17.794 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2021-02-20T08:03:17.832 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libjaeger. 2021-02-20T08:03:17.848 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../001-libjaeger_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:17.890 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libjaeger (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:18.201 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-02-20T08:03:18.216 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../002-liblttng-ust0_2.10.1-1_amd64.deb ... 2021-02-20T08:03:18.285 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2021-02-20T08:03:18.362 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../003-librbd1_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:18.492 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking librbd1 (17.0.0-929-gd318d42d-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-02-20T08:03:18.577 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../003-librbd1_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:18.680 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking librbd1 (17.0.0-929-gd318d42d-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-02-20T08:03:19.185 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../004-librados2_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:19.305 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking librados2 (17.0.0-929-gd318d42d-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-02-20T08:03:19.359 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../004-librados2_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:19.485 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking librados2 (17.0.0-929-gd318d42d-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-02-20T08:03:19.978 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libcephfs2. 2021-02-20T08:03:19.991 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../005-libcephfs2_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:20.036 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libcephfs2 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:20.080 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libcephfs2. 2021-02-20T08:03:20.094 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../005-libcephfs2_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:20.130 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libcephfs2 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:20.364 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-rados. 2021-02-20T08:03:20.379 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../006-python3-rados_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:20.424 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-rados. 2021-02-20T08:03:20.430 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-rados (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:20.438 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../006-python3-rados_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:20.482 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-rados (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:20.767 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-02-20T08:03:20.784 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../007-python3-ceph-argparse_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T08:03:20.825 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-ceph-argparse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:20.835 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-02-20T08:03:20.851 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../007-python3-ceph-argparse_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T08:03:20.894 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-ceph-argparse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:21.079 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-cephfs. 2021-02-20T08:03:21.093 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../008-python3-cephfs_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:21.129 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-cephfs (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:21.137 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-cephfs. 2021-02-20T08:03:21.151 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../008-python3-cephfs_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:21.179 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-cephfs (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:21.440 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-ceph-common. 2021-02-20T08:03:21.454 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../009-python3-ceph-common_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T08:03:21.457 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-ceph-common. 2021-02-20T08:03:21.472 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../009-python3-ceph-common_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T08:03:21.491 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-ceph-common (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:21.516 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-ceph-common (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:21.801 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-prettytable. 2021-02-20T08:03:21.804 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-prettytable. 2021-02-20T08:03:21.818 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../010-python3-prettytable_0.7.2-3_all.deb ... 2021-02-20T08:03:21.819 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../010-python3-prettytable_0.7.2-3_all.deb ... 2021-02-20T08:03:21.860 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2021-02-20T08:03:21.862 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2021-02-20T08:03:22.112 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-rbd. 2021-02-20T08:03:22.123 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-rbd. 2021-02-20T08:03:22.126 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../011-python3-rbd_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:22.138 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../011-python3-rbd_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:22.162 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-rbd (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:22.182 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-rbd (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:22.473 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-02-20T08:03:22.486 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../012-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2021-02-20T08:03:22.515 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-02-20T08:03:22.552 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-02-20T08:03:22.567 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../012-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2021-02-20T08:03:22.610 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-02-20T08:03:22.810 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-02-20T08:03:22.826 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../013-librdkafka1_0.11.3-1build1_amd64.deb ... 2021-02-20T08:03:22.868 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2021-02-20T08:03:22.963 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-02-20T08:03:22.978 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../013-librdkafka1_0.11.3-1build1_amd64.deb ... 2021-02-20T08:03:23.023 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2021-02-20T08:03:23.229 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package librgw2. 2021-02-20T08:03:23.245 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../014-librgw2_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:23.287 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking librgw2 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:23.351 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package librgw2. 2021-02-20T08:03:23.367 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../014-librgw2_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:23.401 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking librgw2 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:23.806 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-rgw. 2021-02-20T08:03:23.822 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../015-python3-rgw_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:23.856 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-rgw (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:23.987 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-rgw. 2021-02-20T08:03:24.001 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../015-python3-rgw_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:24.046 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-rgw (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:24.101 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2021-02-20T08:03:24.117 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../016-libsnappy1v5_1.1.7-1_amd64.deb ... 2021-02-20T08:03:24.151 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2021-02-20T08:03:24.367 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2021-02-20T08:03:24.381 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../016-libsnappy1v5_1.1.7-1_amd64.deb ... 2021-02-20T08:03:24.425 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2021-02-20T08:03:24.453 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2021-02-20T08:03:24.468 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../017-libleveldb1v5_1.20-2_amd64.deb ... 2021-02-20T08:03:24.512 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2021-02-20T08:03:24.737 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2021-02-20T08:03:24.752 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../017-libleveldb1v5_1.20-2_amd64.deb ... 2021-02-20T08:03:24.795 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2021-02-20T08:03:24.823 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package liboath0. 2021-02-20T08:03:24.839 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../018-liboath0_2.6.1-1_amd64.deb ... 2021-02-20T08:03:24.873 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking liboath0 (2.6.1-1) ... 2021-02-20T08:03:25.167 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package liboath0. 2021-02-20T08:03:25.178 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libradosstriper1. 2021-02-20T08:03:25.181 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../018-liboath0_2.6.1-1_amd64.deb ... 2021-02-20T08:03:25.193 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../019-libradosstriper1_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:25.225 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking liboath0 (2.6.1-1) ... 2021-02-20T08:03:25.236 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libradosstriper1 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:25.553 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libradosstriper1. 2021-02-20T08:03:25.568 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../019-libradosstriper1_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:25.603 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libradosstriper1 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:25.680 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-common. 2021-02-20T08:03:25.694 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../020-ceph-common_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:25.730 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-common (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:26.049 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-common. 2021-02-20T08:03:26.062 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../020-ceph-common_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:26.108 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-common (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:27.561 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-base. 2021-02-20T08:03:27.577 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../021-ceph-base_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:27.620 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-base (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:28.082 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-base. 2021-02-20T08:03:28.098 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../021-ceph-base_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:28.149 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-base (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:28.582 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-dateutil. 2021-02-20T08:03:28.595 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../022-python3-dateutil_2.6.1-1_all.deb ... 2021-02-20T08:03:28.632 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2021-02-20T08:03:28.935 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-asn1crypto. 2021-02-20T08:03:28.950 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../023-python3-asn1crypto_0.24.0-1_all.deb ... 2021-02-20T08:03:28.994 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2021-02-20T08:03:29.130 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-dateutil. 2021-02-20T08:03:29.146 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../022-python3-dateutil_2.6.1-1_all.deb ... 2021-02-20T08:03:29.189 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2021-02-20T08:03:29.337 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-cffi-backend. 2021-02-20T08:03:29.352 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../024-python3-cffi-backend_1.11.5-1_amd64.deb ... 2021-02-20T08:03:29.387 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2021-02-20T08:03:29.475 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-asn1crypto. 2021-02-20T08:03:29.491 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../023-python3-asn1crypto_0.24.0-1_all.deb ... 2021-02-20T08:03:29.533 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2021-02-20T08:03:29.657 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-cryptography. 2021-02-20T08:03:29.671 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../025-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2021-02-20T08:03:29.715 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-02-20T08:03:29.812 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-cffi-backend. 2021-02-20T08:03:29.828 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../024-python3-cffi-backend_1.11.5-1_amd64.deb ... 2021-02-20T08:03:29.871 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2021-02-20T08:03:30.018 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-openssl. 2021-02-20T08:03:30.032 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../026-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2021-02-20T08:03:30.076 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2021-02-20T08:03:30.191 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-cryptography. 2021-02-20T08:03:30.204 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../025-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2021-02-20T08:03:30.249 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-02-20T08:03:30.320 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-02-20T08:03:30.334 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../027-ceph-mgr-modules-core_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T08:03:30.370 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr-modules-core (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:30.663 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-openssl. 2021-02-20T08:03:30.677 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../026-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2021-02-20T08:03:30.721 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2021-02-20T08:03:30.734 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-bcrypt. 2021-02-20T08:03:30.748 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../028-python3-bcrypt_3.1.4-2_amd64.deb ... 2021-02-20T08:03:30.791 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2021-02-20T08:03:30.983 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-02-20T08:03:30.999 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../027-ceph-mgr-modules-core_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T08:03:31.041 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-mgr-modules-core (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:31.078 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-cherrypy3. 2021-02-20T08:03:31.093 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../029-python3-cherrypy3_8.9.1-2_all.deb ... 2021-02-20T08:03:31.136 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2021-02-20T08:03:31.406 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-bcrypt. 2021-02-20T08:03:31.421 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../028-python3-bcrypt_3.1.4-2_amd64.deb ... 2021-02-20T08:03:31.464 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2021-02-20T08:03:31.472 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-jwt. 2021-02-20T08:03:31.486 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../030-python3-jwt_1.5.3+ds1-1_all.deb ... 2021-02-20T08:03:31.522 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2021-02-20T08:03:31.784 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-cherrypy3. 2021-02-20T08:03:31.798 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../029-python3-cherrypy3_8.9.1-2_all.deb ... 2021-02-20T08:03:31.825 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-logutils. 2021-02-20T08:03:31.834 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2021-02-20T08:03:31.840 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../031-python3-logutils_0.3.3-5_all.deb ... 2021-02-20T08:03:31.883 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-logutils (0.3.3-5) ... 2021-02-20T08:03:32.204 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-jwt. 2021-02-20T08:03:32.220 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-markupsafe. 2021-02-20T08:03:32.221 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../030-python3-jwt_1.5.3+ds1-1_all.deb ... 2021-02-20T08:03:32.233 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../032-python3-markupsafe_1.0-1build1_amd64.deb ... 2021-02-20T08:03:32.272 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2021-02-20T08:03:32.278 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2021-02-20T08:03:32.572 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-mako. 2021-02-20T08:03:32.576 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-logutils. 2021-02-20T08:03:32.587 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../033-python3-mako_1.0.7+ds1-1_all.deb ... 2021-02-20T08:03:32.590 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../031-python3-logutils_0.3.3-5_all.deb ... 2021-02-20T08:03:32.630 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2021-02-20T08:03:32.634 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-logutils (0.3.3-5) ... 2021-02-20T08:03:32.938 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-markupsafe. 2021-02-20T08:03:32.941 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-simplegeneric. 2021-02-20T08:03:32.952 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../032-python3-markupsafe_1.0-1build1_amd64.deb ... 2021-02-20T08:03:32.956 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../034-python3-simplegeneric_0.8.1-1_all.deb ... 2021-02-20T08:03:32.996 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2021-02-20T08:03:33.017 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2021-02-20T08:03:33.311 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-singledispatch. 2021-02-20T08:03:33.327 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../035-python3-singledispatch_3.4.0.3-2_all.deb ... 2021-02-20T08:03:33.334 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-mako. 2021-02-20T08:03:33.348 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../033-python3-mako_1.0.7+ds1-1_all.deb ... 2021-02-20T08:03:33.369 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2021-02-20T08:03:33.392 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2021-02-20T08:03:33.681 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-webob. 2021-02-20T08:03:33.696 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../036-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2021-02-20T08:03:33.739 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2021-02-20T08:03:33.846 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-simplegeneric. 2021-02-20T08:03:33.859 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../034-python3-simplegeneric_0.8.1-1_all.deb ... 2021-02-20T08:03:33.896 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2021-02-20T08:03:34.044 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-bs4. 2021-02-20T08:03:34.060 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../037-python3-bs4_4.6.0-1_all.deb ... 2021-02-20T08:03:34.102 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2021-02-20T08:03:34.198 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-singledispatch. 2021-02-20T08:03:34.213 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../035-python3-singledispatch_3.4.0.3-2_all.deb ... 2021-02-20T08:03:34.256 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2021-02-20T08:03:34.430 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-waitress. 2021-02-20T08:03:34.445 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../038-python3-waitress_1.0.1-1_all.deb ... 2021-02-20T08:03:34.489 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-waitress (1.0.1-1) ... 2021-02-20T08:03:34.594 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-webob. 2021-02-20T08:03:34.609 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../036-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2021-02-20T08:03:34.652 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2021-02-20T08:03:34.808 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-tempita. 2021-02-20T08:03:34.822 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-2_all.deb ... 2021-02-20T08:03:34.858 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-tempita (0.5.2-2) ... 2021-02-20T08:03:34.956 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-bs4. 2021-02-20T08:03:34.970 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../037-python3-bs4_4.6.0-1_all.deb ... 2021-02-20T08:03:35.015 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2021-02-20T08:03:35.219 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-paste. 2021-02-20T08:03:35.233 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../040-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2021-02-20T08:03:35.286 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-02-20T08:03:35.344 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-waitress. 2021-02-20T08:03:35.358 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../038-python3-waitress_1.0.1-1_all.deb ... 2021-02-20T08:03:35.403 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-waitress (1.0.1-1) ... 2021-02-20T08:03:35.598 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-02-20T08:03:35.612 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2021-02-20T08:03:35.647 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2021-02-20T08:03:35.722 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-tempita. 2021-02-20T08:03:35.737 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-2_all.deb ... 2021-02-20T08:03:35.789 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-tempita (0.5.2-2) ... 2021-02-20T08:03:35.975 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pastedeploy. 2021-02-20T08:03:35.989 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../042-python3-pastedeploy_1.5.2-4_all.deb ... 2021-02-20T08:03:36.042 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2021-02-20T08:03:36.117 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-paste. 2021-02-20T08:03:36.130 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../040-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2021-02-20T08:03:36.175 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-02-20T08:03:36.361 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-webtest. 2021-02-20T08:03:36.378 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../043-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2021-02-20T08:03:36.420 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2021-02-20T08:03:36.488 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-02-20T08:03:36.506 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2021-02-20T08:03:36.547 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2021-02-20T08:03:36.747 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pecan. 2021-02-20T08:03:36.761 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../044-python3-pecan_1.2.1-2_all.deb ... 2021-02-20T08:03:36.806 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pecan (1.2.1-2) ... 2021-02-20T08:03:36.850 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-pastedeploy. 2021-02-20T08:03:36.865 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../042-python3-pastedeploy_1.5.2-4_all.deb ... 2021-02-20T08:03:36.908 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2021-02-20T08:03:37.177 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-werkzeug. 2021-02-20T08:03:37.193 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../045-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2021-02-20T08:03:37.212 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-webtest. 2021-02-20T08:03:37.226 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../043-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2021-02-20T08:03:37.244 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-02-20T08:03:37.271 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2021-02-20T08:03:37.599 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-pecan. 2021-02-20T08:03:37.613 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../044-python3-pecan_1.2.1-2_all.deb ... 2021-02-20T08:03:37.647 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr. 2021-02-20T08:03:37.657 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-pecan (1.2.1-2) ... 2021-02-20T08:03:37.663 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../046-ceph-mgr_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:37.713 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:38.023 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-werkzeug. 2021-02-20T08:03:38.036 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../045-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2021-02-20T08:03:38.081 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-02-20T08:03:38.484 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-mgr. 2021-02-20T08:03:38.500 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../046-ceph-mgr_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:38.543 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-mgr (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:38.884 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mon. 2021-02-20T08:03:38.898 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../047-ceph-mon_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:38.967 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mon (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:39.830 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-mon. 2021-02-20T08:03:39.844 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../047-ceph-mon_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:39.889 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-mon (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:39.929 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-osd. 2021-02-20T08:03:39.942 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../048-ceph-osd_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:39.979 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-osd (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:40.929 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-osd. 2021-02-20T08:03:40.942 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../048-ceph-osd_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:40.987 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-osd (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:42.003 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph. 2021-02-20T08:03:42.017 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../049-ceph_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:42.061 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:42.380 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-fuse. 2021-02-20T08:03:42.396 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../050-ceph-fuse_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:42.439 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-fuse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:42.843 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mds. 2021-02-20T08:03:42.857 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../051-ceph-mds_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:42.926 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mds (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:43.086 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph. 2021-02-20T08:03:43.101 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../049-ceph_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:43.145 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:43.446 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package cephadm. 2021-02-20T08:03:43.464 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../052-cephadm_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:43.481 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-fuse. 2021-02-20T08:03:43.497 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../050-ceph-fuse_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:43.504 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking cephadm (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:43.540 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-fuse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:43.815 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-jinja2. 2021-02-20T08:03:43.829 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../053-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2021-02-20T08:03:43.881 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-02-20T08:03:43.969 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-mds. 2021-02-20T08:03:43.983 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../051-ceph-mds_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:44.028 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-mds (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:44.192 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-02-20T08:03:44.206 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../054-ceph-mgr-cephadm_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T08:03:44.245 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr-cephadm (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:44.539 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package cephadm. 2021-02-20T08:03:44.553 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../052-cephadm_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:03:44.557 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-repoze.lru. 2021-02-20T08:03:44.571 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../055-python3-repoze.lru_0.7-1_all.deb ... 2021-02-20T08:03:44.598 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking cephadm (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:44.607 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2021-02-20T08:03:44.926 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-routes. 2021-02-20T08:03:44.927 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-jinja2. 2021-02-20T08:03:44.940 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../053-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2021-02-20T08:03:44.942 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../056-python3-routes_2.4.1-1_all.deb ... 2021-02-20T08:03:44.985 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-routes (2.4.1-1) ... 2021-02-20T08:03:44.985 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-02-20T08:03:45.322 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-02-20T08:03:45.324 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-02-20T08:03:45.336 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T08:03:45.340 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../054-ceph-mgr-cephadm_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T08:03:45.383 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-mgr-cephadm (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:45.388 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr-dashboard (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:45.713 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-repoze.lru. 2021-02-20T08:03:45.728 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../055-python3-repoze.lru_0.7-1_all.deb ... 2021-02-20T08:03:45.771 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2021-02-20T08:03:46.109 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-routes. 2021-02-20T08:03:46.124 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../056-python3-routes_2.4.1-1_all.deb ... 2021-02-20T08:03:46.167 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-routes (2.4.1-1) ... 2021-02-20T08:03:46.312 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-numpy. 2021-02-20T08:03:46.326 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../058-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2021-02-20T08:03:46.378 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2021-02-20T08:03:46.505 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-02-20T08:03:46.519 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T08:03:46.555 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-mgr-dashboard (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:47.001 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-decorator. 2021-02-20T08:03:47.014 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../059-python3-decorator_4.1.2-1_all.deb ... 2021-02-20T08:03:47.059 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-decorator (4.1.2-1) ... 2021-02-20T08:03:47.378 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-scipy. 2021-02-20T08:03:47.393 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../060-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2021-02-20T08:03:47.416 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-numpy. 2021-02-20T08:03:47.430 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../058-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2021-02-20T08:03:47.445 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2021-02-20T08:03:47.475 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2021-02-20T08:03:48.115 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-decorator. 2021-02-20T08:03:48.128 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../059-python3-decorator_4.1.2-1_all.deb ... 2021-02-20T08:03:48.173 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-decorator (4.1.2-1) ... 2021-02-20T08:03:48.477 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-scipy. 2021-02-20T08:03:48.492 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../060-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2021-02-20T08:03:48.553 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2021-02-20T08:03:49.182 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-02-20T08:03:49.197 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../061-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2021-02-20T08:03:49.248 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2021-02-20T08:03:49.714 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-joblib. 2021-02-20T08:03:49.729 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../062-python3-joblib_0.11-1_all.deb ... 2021-02-20T08:03:49.772 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-joblib (0.11-1) ... 2021-02-20T08:03:50.040 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-02-20T08:03:50.055 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../061-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2021-02-20T08:03:50.098 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2021-02-20T08:03:50.127 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-sklearn. 2021-02-20T08:03:50.141 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../063-python3-sklearn_0.19.1-3_all.deb ... 2021-02-20T08:03:50.185 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2021-02-20T08:03:50.607 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-joblib. 2021-02-20T08:03:50.621 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../062-python3-joblib_0.11-1_all.deb ... 2021-02-20T08:03:50.667 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-joblib (0.11-1) ... 2021-02-20T08:03:50.703 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-02-20T08:03:50.717 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../064-ceph-mgr-diskprediction-local_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T08:03:50.761 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr-diskprediction-local (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:51.006 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-sklearn. 2021-02-20T08:03:51.019 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../063-python3-sklearn_0.19.1-3_all.deb ... 2021-02-20T08:03:51.056 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2021-02-20T08:03:51.643 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-02-20T08:03:51.657 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../064-ceph-mgr-diskprediction-local_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T08:03:51.701 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-mgr-diskprediction-local (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:51.877 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-monotonic. 2021-02-20T08:03:51.891 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../065-python3-monotonic_1.1-2_all.deb ... 2021-02-20T08:03:51.935 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-monotonic (1.1-2) ... 2021-02-20T08:03:52.213 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-fasteners. 2021-02-20T08:03:52.230 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../066-python3-fasteners_0.12.0-3_all.deb ... 2021-02-20T08:03:52.271 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2021-02-20T08:03:52.608 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-httplib2. 2021-02-20T08:03:52.622 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../067-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2021-02-20T08:03:52.666 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2021-02-20T08:03:52.869 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-monotonic. 2021-02-20T08:03:52.882 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../065-python3-monotonic_1.1-2_all.deb ... 2021-02-20T08:03:52.919 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-monotonic (1.1-2) ... 2021-02-20T08:03:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pyasn1. 2021-02-20T08:03:53.033 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../068-python3-pyasn1_0.4.2-3_all.deb ... 2021-02-20T08:03:53.077 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2021-02-20T08:03:53.213 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-fasteners. 2021-02-20T08:03:53.229 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../066-python3-fasteners_0.12.0-3_all.deb ... 2021-02-20T08:03:53.272 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2021-02-20T08:03:53.397 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-02-20T08:03:53.413 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../069-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2021-02-20T08:03:53.463 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2021-02-20T08:03:53.593 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-httplib2. 2021-02-20T08:03:53.608 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../067-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2021-02-20T08:03:53.651 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2021-02-20T08:03:53.775 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-rsa. 2021-02-20T08:03:53.789 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../070-python3-rsa_3.4.2-1_all.deb ... 2021-02-20T08:03:53.833 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-rsa (3.4.2-1) ... 2021-02-20T08:03:54.004 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-pyasn1. 2021-02-20T08:03:54.018 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../068-python3-pyasn1_0.4.2-3_all.deb ... 2021-02-20T08:03:54.062 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2021-02-20T08:03:54.145 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-sqlalchemy. 2021-02-20T08:03:54.158 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../071-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2021-02-20T08:03:54.203 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-02-20T08:03:54.401 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-02-20T08:03:54.414 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../069-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2021-02-20T08:03:54.459 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2021-02-20T08:03:54.624 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-oauth2client. 2021-02-20T08:03:54.638 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../072-python3-oauth2client_4.1.2-3_all.deb ... 2021-02-20T08:03:54.673 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2021-02-20T08:03:54.762 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-rsa. 2021-02-20T08:03:54.779 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../070-python3-rsa_3.4.2-1_all.deb ... 2021-02-20T08:03:54.820 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-rsa (3.4.2-1) ... 2021-02-20T08:03:54.943 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-setuptools. 2021-02-20T08:03:54.958 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../073-python3-setuptools_39.0.1-2_all.deb ... 2021-02-20T08:03:54.993 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2021-02-20T08:03:55.132 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-sqlalchemy. 2021-02-20T08:03:55.148 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../071-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2021-02-20T08:03:55.191 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-02-20T08:03:55.321 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-websocket. 2021-02-20T08:03:55.335 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../074-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2021-02-20T08:03:55.371 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2021-02-20T08:03:55.633 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-kubernetes. 2021-02-20T08:03:55.646 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../075-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2021-02-20T08:03:55.664 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-oauth2client. 2021-02-20T08:03:55.674 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2021-02-20T08:03:55.679 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../072-python3-oauth2client_4.1.2-3_all.deb ... 2021-02-20T08:03:55.722 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2021-02-20T08:03:56.050 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-setuptools. 2021-02-20T08:03:56.065 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../073-python3-setuptools_39.0.1-2_all.deb ... 2021-02-20T08:03:56.078 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-02-20T08:03:56.092 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T08:03:56.108 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2021-02-20T08:03:56.137 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr-k8sevents (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:56.456 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-json-pointer. 2021-02-20T08:03:56.470 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../077-python3-json-pointer_1.10-1_all.deb ... 2021-02-20T08:03:56.514 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-json-pointer (1.10-1) ... 2021-02-20T08:03:56.555 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-websocket. 2021-02-20T08:03:56.570 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../074-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2021-02-20T08:03:56.613 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2021-02-20T08:03:56.876 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-jsonpatch. 2021-02-20T08:03:56.893 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../078-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2021-02-20T08:03:56.925 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-kubernetes. 2021-02-20T08:03:56.937 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-02-20T08:03:56.940 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../075-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2021-02-20T08:03:56.984 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2021-02-20T08:03:57.239 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr-rook. 2021-02-20T08:03:57.254 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../079-ceph-mgr-rook_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T08:03:57.298 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr-rook (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:57.431 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-02-20T08:03:57.446 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T08:03:57.489 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-mgr-k8sevents (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:57.667 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libonig4:amd64. 2021-02-20T08:03:57.681 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../080-libonig4_6.7.0-1_amd64.deb ... 2021-02-20T08:03:57.819 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2021-02-20T08:03:57.834 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-json-pointer. 2021-02-20T08:03:57.849 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../077-python3-json-pointer_1.10-1_all.deb ... 2021-02-20T08:03:57.892 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-json-pointer (1.10-1) ... 2021-02-20T08:03:58.172 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libjq1:amd64. 2021-02-20T08:03:58.189 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../081-libjq1_1.5+dfsg-2_amd64.deb ... 2021-02-20T08:03:58.222 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-jsonpatch. 2021-02-20T08:03:58.230 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2021-02-20T08:03:58.238 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../078-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2021-02-20T08:03:58.283 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-02-20T08:03:58.475 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package jq. 2021-02-20T08:03:58.489 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../082-jq_1.5+dfsg-2_amd64.deb ... 2021-02-20T08:03:58.533 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking jq (1.5+dfsg-2) ... 2021-02-20T08:03:58.586 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-mgr-rook. 2021-02-20T08:03:58.601 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../079-ceph-mgr-rook_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T08:03:58.644 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-mgr-rook (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:03:58.819 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package socat. 2021-02-20T08:03:58.835 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../083-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2021-02-20T08:03:58.877 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2021-02-20T08:03:59.015 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libonig4:amd64. 2021-02-20T08:03:59.030 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../080-libonig4_6.7.0-1_amd64.deb ... 2021-02-20T08:03:59.159 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2021-02-20T08:03:59.264 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-02-20T08:03:59.278 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../084-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2021-02-20T08:03:59.322 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-02-20T08:03:59.513 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libjq1:amd64. 2021-02-20T08:03:59.528 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../081-libjq1_1.5+dfsg-2_amd64.deb ... 2021-02-20T08:03:59.571 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2021-02-20T08:03:59.600 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package xmlstarlet. 2021-02-20T08:03:59.616 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2_amd64.deb ... 2021-02-20T08:03:59.658 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2021-02-20T08:03:59.859 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package jq. 2021-02-20T08:03:59.872 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../082-jq_1.5+dfsg-2_amd64.deb ... 2021-02-20T08:03:59.917 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking jq (1.5+dfsg-2) ... 2021-02-20T08:04:00.012 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-test. 2021-02-20T08:04:00.028 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../086-ceph-test_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:04:00.073 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-test (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:00.195 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package socat. 2021-02-20T08:04:00.209 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../083-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2021-02-20T08:04:00.253 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2021-02-20T08:04:00.657 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-02-20T08:04:00.673 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../084-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2021-02-20T08:04:00.716 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-02-20T08:04:01.003 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package xmlstarlet. 2021-02-20T08:04:01.018 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2_amd64.deb ... 2021-02-20T08:04:01.061 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2021-02-20T08:04:01.450 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-test. 2021-02-20T08:04:01.465 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../086-ceph-test_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:04:01.508 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-test (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:03.467 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libcephfs-dev. 2021-02-20T08:04:03.482 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../087-libcephfs-dev_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:04:03.525 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libcephfs-dev (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:03.835 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package nvme-cli. 2021-02-20T08:04:03.843 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../088-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2021-02-20T08:04:03.886 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2021-02-20T08:04:04.232 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python-attr. 2021-02-20T08:04:04.246 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../089-python-attr_17.4.0-2_all.deb ... 2021-02-20T08:04:04.290 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python-attr (17.4.0-2) ... 2021-02-20T08:04:04.626 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python-funcsigs. 2021-02-20T08:04:04.641 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../090-python-funcsigs_1.0.2-4_all.deb ... 2021-02-20T08:04:04.693 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2021-02-20T08:04:04.890 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libcephfs-dev. 2021-02-20T08:04:04.905 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../087-libcephfs-dev_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:04:04.948 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libcephfs-dev (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:05.020 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python-pluggy. 2021-02-20T08:04:05.034 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../091-python-pluggy_0.6.0-1_all.deb ... 2021-02-20T08:04:05.079 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python-pluggy (0.6.0-1) ... 2021-02-20T08:04:05.243 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package nvme-cli. 2021-02-20T08:04:05.257 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../088-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2021-02-20T08:04:05.302 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2021-02-20T08:04:05.390 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python-py. 2021-02-20T08:04:05.405 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../092-python-py_1.5.2-1_all.deb ... 2021-02-20T08:04:05.449 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python-py (1.5.2-1) ... 2021-02-20T08:04:05.648 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python-attr. 2021-02-20T08:04:05.662 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../089-python-attr_17.4.0-2_all.deb ... 2021-02-20T08:04:05.698 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python-attr (17.4.0-2) ... 2021-02-20T08:04:05.760 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python-pytest. 2021-02-20T08:04:05.776 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../093-python-pytest_3.3.2-2_all.deb ... 2021-02-20T08:04:05.819 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python-pytest (3.3.2-2) ... 2021-02-20T08:04:06.002 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python-funcsigs. 2021-02-20T08:04:06.017 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../090-python-funcsigs_1.0.2-4_all.deb ... 2021-02-20T08:04:06.060 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2021-02-20T08:04:06.147 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-attr. 2021-02-20T08:04:06.161 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../094-python3-attr_17.4.0-2_all.deb ... 2021-02-20T08:04:06.205 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-attr (17.4.0-2) ... 2021-02-20T08:04:06.373 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python-pluggy. 2021-02-20T08:04:06.388 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../091-python-pluggy_0.6.0-1_all.deb ... 2021-02-20T08:04:06.431 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python-pluggy (0.6.0-1) ... 2021-02-20T08:04:06.516 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-cycler. 2021-02-20T08:04:06.532 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../095-python3-cycler_0.10.0-1_all.deb ... 2021-02-20T08:04:06.575 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-cycler (0.10.0-1) ... 2021-02-20T08:04:06.743 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python-py. 2021-02-20T08:04:06.759 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../092-python-py_1.5.2-1_all.deb ... 2021-02-20T08:04:06.801 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python-py (1.5.2-1) ... 2021-02-20T08:04:06.861 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-webencodings. 2021-02-20T08:04:06.878 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../096-python3-webencodings_0.5-2_all.deb ... 2021-02-20T08:04:06.919 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-webencodings (0.5-2) ... 2021-02-20T08:04:07.097 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python-pytest. 2021-02-20T08:04:07.114 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../093-python-pytest_3.3.2-2_all.deb ... 2021-02-20T08:04:07.156 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python-pytest (3.3.2-2) ... 2021-02-20T08:04:07.213 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-html5lib. 2021-02-20T08:04:07.227 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../097-python3-html5lib_0.999999999-1_all.deb ... 2021-02-20T08:04:07.280 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2021-02-20T08:04:07.476 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-attr. 2021-02-20T08:04:07.494 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../094-python3-attr_17.4.0-2_all.deb ... 2021-02-20T08:04:07.534 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-attr (17.4.0-2) ... 2021-02-20T08:04:07.641 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-02-20T08:04:07.658 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../098-python3-lxml_4.2.1-1ubuntu0.3_amd64.deb ... 2021-02-20T08:04:07.699 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-02-20T08:04:07.829 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-cycler. 2021-02-20T08:04:07.846 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../095-python3-cycler_0.10.0-1_all.deb ... 2021-02-20T08:04:07.889 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-cycler (0.10.0-1) ... 2021-02-20T08:04:08.145 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pyparsing. 2021-02-20T08:04:08.160 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../099-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2021-02-20T08:04:08.184 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-webencodings. 2021-02-20T08:04:08.198 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../096-python3-webencodings_0.5-2_all.deb ... 2021-02-20T08:04:08.211 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2021-02-20T08:04:08.242 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-webencodings (0.5-2) ... 2021-02-20T08:04:08.522 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-tz. 2021-02-20T08:04:08.529 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-html5lib. 2021-02-20T08:04:08.537 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../100-python3-tz_2018.3-2_all.deb ... 2021-02-20T08:04:08.542 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../097-python3-html5lib_0.999999999-1_all.deb ... 2021-02-20T08:04:08.580 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-tz (2018.3-2) ... 2021-02-20T08:04:08.587 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2021-02-20T08:04:08.926 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-02-20T08:04:08.934 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-matplotlib. 2021-02-20T08:04:08.943 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../098-python3-lxml_4.2.1-1ubuntu0.3_amd64.deb ... 2021-02-20T08:04:08.948 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../101-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2021-02-20T08:04:08.983 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2021-02-20T08:04:08.984 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-02-20T08:04:09.429 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-pyparsing. 2021-02-20T08:04:09.445 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../099-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2021-02-20T08:04:09.488 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2021-02-20T08:04:09.800 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-tz. 2021-02-20T08:04:09.818 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../100-python3-tz_2018.3-2_all.deb ... 2021-02-20T08:04:09.823 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-olefile. 2021-02-20T08:04:09.838 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../102-python3-olefile_0.45.1-1_all.deb ... 2021-02-20T08:04:09.859 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-tz (2018.3-2) ... 2021-02-20T08:04:09.882 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-olefile (0.45.1-1) ... 2021-02-20T08:04:10.156 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-matplotlib. 2021-02-20T08:04:10.169 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../101-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2021-02-20T08:04:10.176 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pastescript. 2021-02-20T08:04:10.194 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-2_all.deb ... 2021-02-20T08:04:10.212 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2021-02-20T08:04:10.234 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2021-02-20T08:04:10.597 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pil:amd64. 2021-02-20T08:04:10.614 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../104-python3-pil_5.1.0-1ubuntu0.4_amd64.deb ... 2021-02-20T08:04:10.655 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.4) ... 2021-02-20T08:04:10.912 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-olefile. 2021-02-20T08:04:10.925 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../102-python3-olefile_0.45.1-1_all.deb ... 2021-02-20T08:04:10.969 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-olefile (0.45.1-1) ... 2021-02-20T08:04:11.067 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pluggy. 2021-02-20T08:04:11.081 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../105-python3-pluggy_0.6.0-1_all.deb ... 2021-02-20T08:04:11.125 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2021-02-20T08:04:11.307 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-pastescript. 2021-02-20T08:04:11.321 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-2_all.deb ... 2021-02-20T08:04:11.365 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2021-02-20T08:04:11.445 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-py. 2021-02-20T08:04:11.464 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../106-python3-py_1.5.2-1_all.deb ... 2021-02-20T08:04:11.503 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-py (1.5.2-1) ... 2021-02-20T08:04:11.787 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-pil:amd64. 2021-02-20T08:04:11.801 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../104-python3-pil_5.1.0-1ubuntu0.4_amd64.deb ... 2021-02-20T08:04:11.831 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pyinotify. 2021-02-20T08:04:11.836 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.4) ... 2021-02-20T08:04:11.845 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1_all.deb ... 2021-02-20T08:04:11.889 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2021-02-20T08:04:12.190 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-pluggy. 2021-02-20T08:04:12.193 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pytest. 2021-02-20T08:04:12.205 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../105-python3-pluggy_0.6.0-1_all.deb ... 2021-02-20T08:04:12.206 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../108-python3-pytest_3.3.2-2_all.deb ... 2021-02-20T08:04:12.248 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2021-02-20T08:04:12.250 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pytest (3.3.2-2) ... 2021-02-20T08:04:12.560 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-py. 2021-02-20T08:04:12.574 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../106-python3-py_1.5.2-1_all.deb ... 2021-02-20T08:04:12.603 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-simplejson. 2021-02-20T08:04:12.618 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-py (1.5.2-1) ... 2021-02-20T08:04:12.619 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../109-python3-simplejson_3.13.2-1_amd64.deb ... 2021-02-20T08:04:12.661 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2021-02-20T08:04:12.940 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-pyinotify. 2021-02-20T08:04:12.954 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1_all.deb ... 2021-02-20T08:04:12.998 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2021-02-20T08:04:13.023 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-tk:amd64. 2021-02-20T08:04:13.040 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../110-python3-tk_3.6.9-1~18.04_amd64.deb ... 2021-02-20T08:04:13.081 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2021-02-20T08:04:13.326 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-pytest. 2021-02-20T08:04:13.340 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../108-python3-pytest_3.3.2-2_all.deb ... 2021-02-20T08:04:13.384 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-pytest (3.3.2-2) ... 2021-02-20T08:04:13.552 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package radosgw. 2021-02-20T08:04:13.566 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../111-radosgw_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:04:13.610 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking radosgw (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:13.706 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-simplejson. 2021-02-20T08:04:13.721 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../109-python3-simplejson_3.13.2-1_amd64.deb ... 2021-02-20T08:04:13.764 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2021-02-20T08:04:14.092 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-tk:amd64. 2021-02-20T08:04:14.106 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../110-python3-tk_3.6.9-1~18.04_amd64.deb ... 2021-02-20T08:04:14.176 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2021-02-20T08:04:14.639 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package radosgw. 2021-02-20T08:04:14.656 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../111-radosgw_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:04:14.698 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking radosgw (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:14.755 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package rbd-fuse. 2021-02-20T08:04:14.772 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../112-rbd-fuse_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:04:14.813 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking rbd-fuse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:15.183 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package rbd-mirror. 2021-02-20T08:04:15.199 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../113-rbd-mirror_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:04:15.241 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking rbd-mirror (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:15.769 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package rbd-fuse. 2021-02-20T08:04:15.782 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../112-rbd-fuse_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:04:15.826 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking rbd-fuse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:15.884 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-logutils (0.3.3-5) ... 2021-02-20T08:04:16.163 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package rbd-mirror. 2021-02-20T08:04:16.179 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../113-rbd-mirror_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T08:04:16.200 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-02-20T08:04:16.221 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking rbd-mirror (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:16.564 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2021-02-20T08:04:16.690 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-02-20T08:04:16.901 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-logutils (0.3.3-5) ... 2021-02-20T08:04:17.211 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-02-20T08:04:17.326 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.4) ... 2021-02-20T08:04:17.605 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2021-02-20T08:04:17.719 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2021-02-20T08:04:17.731 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-02-20T08:04:18.003 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-webencodings (0.5-2) ... 2021-02-20T08:04:18.296 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python-attr (17.4.0-2) ... 2021-02-20T08:04:18.368 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.4) ... 2021-02-20T08:04:18.497 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2021-02-20T08:04:18.624 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-py (1.5.2-1) ... 2021-02-20T08:04:18.745 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2021-02-20T08:04:18.967 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-waitress (1.0.1-1) ... 2021-02-20T08:04:19.038 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-webencodings (0.5-2) ... 2021-02-20T08:04:19.331 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python-attr (17.4.0-2) ... 2021-02-20T08:04:19.439 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2021-02-20T08:04:19.533 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2021-02-20T08:04:19.660 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-py (1.5.2-1) ... 2021-02-20T08:04:20.003 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-waitress (1.0.1-1) ... 2021-02-20T08:04:20.128 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-olefile (0.45.1-1) ... 2021-02-20T08:04:20.294 INFO:teuthology.orchestra.run.smithi090.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2021-02-20T08:04:20.437 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-ceph-argparse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:20.590 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-olefile (0.45.1-1) ... 2021-02-20T08:04:20.738 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-tempita (0.5.2-2) ... 2021-02-20T08:04:20.883 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-ceph-argparse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:21.065 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2021-02-20T08:04:21.185 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-tempita (0.5.2-2) ... 2021-02-20T08:04:21.191 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2021-02-20T08:04:21.503 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2021-02-20T08:04:21.638 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2021-02-20T08:04:22.152 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2021-02-20T08:04:22.470 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2021-02-20T08:04:22.583 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2021-02-20T08:04:22.746 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python-py (1.5.2-1) ... 2021-02-20T08:04:22.901 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2021-02-20T08:04:22.981 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-simplejson (3.13.2-1) ... 2021-02-20T08:04:23.195 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python-py (1.5.2-1) ... 2021-02-20T08:04:23.318 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-json-pointer (1.10-1) ... 2021-02-20T08:04:23.406 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2021-02-20T08:04:23.422 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-simplejson (3.13.2-1) ... 2021-02-20T08:04:23.636 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2021-02-20T08:04:23.763 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python-pluggy (0.6.0-1) ... 2021-02-20T08:04:23.773 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-json-pointer (1.10-1) ... 2021-02-20T08:04:23.863 INFO:teuthology.orchestra.run.smithi090.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2021-02-20T08:04:23.955 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2021-02-20T08:04:24.093 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2021-02-20T08:04:24.219 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python-pluggy (0.6.0-1) ... 2021-02-20T08:04:24.248 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2021-02-20T08:04:24.430 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2021-02-20T08:04:24.566 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-cycler (0.10.0-1) ... 2021-02-20T08:04:24.909 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2021-02-20T08:04:25.223 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2021-02-20T08:04:25.285 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-bs4 (4.6.0-1) ... 2021-02-20T08:04:25.533 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-cycler (0.10.0-1) ... 2021-02-20T08:04:25.620 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pluggy (0.6.0-1) ... 2021-02-20T08:04:25.817 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2021-02-20T08:04:25.930 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up cephadm (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:26.177 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-bs4 (4.6.0-1) ... 2021-02-20T08:04:26.512 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-pluggy (0.6.0-1) ... 2021-02-20T08:04:26.582 INFO:teuthology.orchestra.run.smithi003.stdout:Adding system user cephadm....done 2021-02-20T08:04:26.770 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2021-02-20T08:04:26.806 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up cephadm (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:27.080 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-02-20T08:04:27.206 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2021-02-20T08:04:27.350 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2021-02-20T08:04:27.441 INFO:teuthology.orchestra.run.smithi090.stdout:Adding system user cephadm....done 2021-02-20T08:04:27.572 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2021-02-20T08:04:27.708 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-prettytable (0.7.2-3) ... 2021-02-20T08:04:27.858 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-02-20T08:04:27.984 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2021-02-20T08:04:28.017 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2021-02-20T08:04:28.118 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2021-02-20T08:04:28.198 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2021-02-20T08:04:28.445 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-prettytable (0.7.2-3) ... 2021-02-20T08:04:28.503 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-ceph-common (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:28.721 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2021-02-20T08:04:28.813 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-monotonic (1.1-2) ... 2021-02-20T08:04:28.822 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2021-02-20T08:04:29.114 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2021-02-20T08:04:29.142 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-ceph-common (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:29.240 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-02-20T08:04:29.460 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-monotonic (1.1-2) ... 2021-02-20T08:04:29.600 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-html5lib (0.999999999-1) ... 2021-02-20T08:04:29.778 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2021-02-20T08:04:29.887 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-02-20T08:04:29.968 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-02-20T08:04:30.066 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2021-02-20T08:04:30.098 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2021-02-20T08:04:30.205 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-html5lib (0.999999999-1) ... 2021-02-20T08:04:30.328 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-repoze.lru (0.7-1) ... 2021-02-20T08:04:30.524 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-02-20T08:04:30.615 INFO:teuthology.orchestra.run.smithi090.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2021-02-20T08:04:30.639 INFO:teuthology.orchestra.run.smithi090.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2021-02-20T08:04:30.646 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2021-02-20T08:04:30.877 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-repoze.lru (0.7-1) ... 2021-02-20T08:04:30.968 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2021-02-20T08:04:31.164 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2021-02-20T08:04:31.309 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-attr (17.4.0-2) ... 2021-02-20T08:04:31.431 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2021-02-20T08:04:31.618 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python-funcsigs (1.0.2-4) ... 2021-02-20T08:04:31.750 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-attr (17.4.0-2) ... 2021-02-20T08:04:31.827 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libjaeger (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:31.954 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-02-20T08:04:32.054 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python-funcsigs (1.0.2-4) ... 2021-02-20T08:04:32.270 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libjaeger (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:32.363 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2021-02-20T08:04:32.380 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-02-20T08:04:32.454 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-02-20T08:04:32.717 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up liboath0 (2.6.1-1) ... 2021-02-20T08:04:32.774 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2021-02-20T08:04:32.855 INFO:teuthology.orchestra.run.smithi090.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-02-20T08:04:32.892 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-02-20T08:04:33.035 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2021-02-20T08:04:33.104 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up liboath0 (2.6.1-1) ... 2021-02-20T08:04:33.287 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-02-20T08:04:33.303 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2021-02-20T08:04:33.396 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2021-02-20T08:04:33.430 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-setuptools (39.0.1-2) ... 2021-02-20T08:04:33.614 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2021-02-20T08:04:33.725 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-setuptools (39.0.1-2) ... 2021-02-20T08:04:33.840 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2021-02-20T08:04:33.968 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-decorator (4.1.2-1) ... 2021-02-20T08:04:34.118 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2021-02-20T08:04:34.243 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-tz (2018.3-2) ... 2021-02-20T08:04:34.247 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-decorator (4.1.2-1) ... 2021-02-20T08:04:34.515 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-tz (2018.3-2) ... 2021-02-20T08:04:34.519 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2021-02-20T08:04:34.777 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2021-02-20T08:04:34.837 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2021-02-20T08:04:35.042 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2021-02-20T08:04:35.147 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-joblib (0.11-1) ... 2021-02-20T08:04:35.335 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-joblib (0.11-1) ... 2021-02-20T08:04:35.515 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-routes (2.4.1-1) ... 2021-02-20T08:04:35.703 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-routes (2.4.1-1) ... 2021-02-20T08:04:35.816 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2021-02-20T08:04:35.998 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2021-02-20T08:04:36.109 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-dateutil (2.6.1-1) ... 2021-02-20T08:04:36.282 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-dateutil (2.6.1-1) ... 2021-02-20T08:04:36.410 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-02-20T08:04:36.575 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-02-20T08:04:36.754 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-fasteners (0.12.0-3) ... 2021-02-20T08:04:36.936 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-fasteners (0.12.0-3) ... 2021-02-20T08:04:37.064 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up jq (1.5+dfsg-2) ... 2021-02-20T08:04:37.198 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2021-02-20T08:04:37.237 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up jq (1.5+dfsg-2) ... 2021-02-20T08:04:37.355 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2021-02-20T08:04:38.412 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-02-20T08:04:38.530 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-02-20T08:04:38.714 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2021-02-20T08:04:38.824 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2021-02-20T08:04:39.065 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2021-02-20T08:04:39.117 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2021-02-20T08:04:39.182 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pytest (3.3.2-2) ... 2021-02-20T08:04:39.244 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-pytest (3.3.2-2) ... 2021-02-20T08:04:39.534 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python-pytest (3.3.2-2) ... 2021-02-20T08:04:39.595 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python-pytest (3.3.2-2) ... 2021-02-20T08:04:39.769 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2021-02-20T08:04:39.846 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2021-02-20T08:04:39.886 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up xmlstarlet (1.6.1-2) ... 2021-02-20T08:04:39.956 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up xmlstarlet (1.6.1-2) ... 2021-02-20T08:04:39.987 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up librados2 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:40.065 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up librados2 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:40.113 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-rsa (3.4.2-1) ... 2021-02-20T08:04:40.167 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-rsa (3.4.2-1) ... 2021-02-20T08:04:40.439 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libcephfs2 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:40.469 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libcephfs2 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:40.590 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libcephfs-dev (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:40.595 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libcephfs-dev (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:40.700 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-rados (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:40.721 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-rados (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:40.801 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2021-02-20T08:04:40.847 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2021-02-20T08:04:41.093 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2021-02-20T08:04:41.123 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2021-02-20T08:04:41.378 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2021-02-20T08:04:41.408 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2021-02-20T08:04:41.781 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2021-02-20T08:04:41.796 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2021-02-20T08:04:42.649 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-sklearn (0.19.1-3) ... 2021-02-20T08:04:42.667 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-sklearn (0.19.1-3) ... 2021-02-20T08:04:43.498 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2021-02-20T08:04:43.505 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2021-02-20T08:04:43.774 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libradosstriper1 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:43.798 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libradosstriper1 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:43.888 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-cephfs (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:43.902 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-cephfs (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:44.003 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up librgw2 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:44.014 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up librgw2 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:44.104 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-fuse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:44.123 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-fuse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:44.326 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-02-20T08:04:44.327 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-02-20T08:04:44.328 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-02-20T08:04:44.328 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-02-20T08:04:44.575 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up librbd1 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:44.597 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up librbd1 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:44.676 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-pecan (1.2.1-2) ... 2021-02-20T08:04:44.740 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pecan (1.2.1-2) ... 2021-02-20T08:04:44.749 INFO:teuthology.orchestra.run.smithi090.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2021-02-20T08:04:44.781 INFO:teuthology.orchestra.run.smithi090.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2021-02-20T08:04:44.829 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2021-02-20T08:04:44.862 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2021-02-20T08:04:45.098 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2021-02-20T08:04:45.177 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2021-02-20T08:04:45.767 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-mgr-modules-core (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:45.837 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr-modules-core (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:45.868 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-pastescript (2.0.2-2) ... 2021-02-20T08:04:45.955 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pastescript (2.0.2-2) ... 2021-02-20T08:04:46.170 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-rgw (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:46.273 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-rgw (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:46.296 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up rbd-fuse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:46.399 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up rbd-fuse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:46.423 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-rbd (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:46.524 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-common (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:46.533 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-rbd (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:46.634 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-common (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:46.984 INFO:teuthology.orchestra.run.smithi090.stdout:Adding group ceph....done 2021-02-20T08:04:47.094 INFO:teuthology.orchestra.run.smithi003.stdout:Adding group ceph....done 2021-02-20T08:04:47.338 INFO:teuthology.orchestra.run.smithi090.stdout:Adding system user ceph....done 2021-02-20T08:04:47.440 INFO:teuthology.orchestra.run.smithi003.stdout:Adding system user ceph....done 2021-02-20T08:04:47.506 INFO:teuthology.orchestra.run.smithi090.stdout:Setting system user ceph properties....done 2021-02-20T08:04:47.606 INFO:teuthology.orchestra.run.smithi003.stdout:Setting system user ceph properties....done 2021-02-20T08:04:47.660 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-02-20T08:04:47.762 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-02-20T08:04:47.881 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-02-20T08:04:47.978 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-02-20T08:04:48.203 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up radosgw (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:48.261 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up radosgw (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:48.475 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-02-20T08:04:48.476 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-02-20T08:04:48.501 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-02-20T08:04:48.501 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-02-20T08:04:48.872 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up rbd-mirror (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:48.905 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up rbd-mirror (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:49.083 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2021-02-20T08:04:49.083 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2021-02-20T08:04:49.124 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2021-02-20T08:04:49.125 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2021-02-20T08:04:49.374 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-test (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:49.381 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-test (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:49.484 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-base (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:49.507 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-base (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:49.783 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-02-20T08:04:49.838 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-02-20T08:04:50.045 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-mgr (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:50.263 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-02-20T08:04:50.263 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-02-20T08:04:50.532 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-mgr-diskprediction-local (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:50.834 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:50.834 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-02-20T08:04:50.834 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-02-20T08:04:50.835 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-osd (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:51.045 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr-diskprediction-local (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:51.099 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-02-20T08:04:51.099 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-02-20T08:04:51.163 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-osd (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:51.353 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-mgr-cephadm (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:51.479 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-mgr-rook (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:51.614 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-mds (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:51.617 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-02-20T08:04:51.617 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-02-20T08:04:51.841 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-02-20T08:04:51.841 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-02-20T08:04:51.890 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr-cephadm (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:52.041 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr-rook (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:52.115 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-mon (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:52.159 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mds (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:52.333 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-02-20T08:04:52.333 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-02-20T08:04:52.369 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-02-20T08:04:52.369 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-02-20T08:04:52.617 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-mgr-k8sevents (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:52.644 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mon (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:52.718 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-mgr-dashboard (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:52.844 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-02-20T08:04:52.845 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-02-20T08:04:52.845 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:52.964 INFO:teuthology.orchestra.run.smithi090.stdout:Processing triggers for systemd (237-3ubuntu10.44) ... 2021-02-20T08:04:53.128 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr-k8sevents (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:53.148 INFO:teuthology.orchestra.run.smithi090.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-02-20T08:04:53.229 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr-dashboard (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:53.355 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T08:04:53.482 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for systemd (237-3ubuntu10.44) ... 2021-02-20T08:04:53.683 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-02-20T08:04:55.218 INFO:teuthology.orchestra.run.smithi090.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2021-02-20T08:04:55.253 INFO:teuthology.orchestra.run.smithi090.stdout:ureadahead will be reprofiled on next reboot 2021-02-20T08:04:55.301 INFO:teuthology.orchestra.run.smithi090.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.4) ... 2021-02-20T08:04:55.786 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2021-02-20T08:04:55.820 INFO:teuthology.orchestra.run.smithi003.stdout:ureadahead will be reprofiled on next reboot 2021-02-20T08:04:55.870 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.4) ... 2021-02-20T08:04:56.298 INFO:teuthology.orchestra.run.smithi003.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-02-20T08:04:56.300 DEBUG:teuthology.parallel:result is None 2021-02-20T08:04:56.418 INFO:teuthology.orchestra.run.smithi090.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-02-20T08:04:56.421 DEBUG:teuthology.parallel:result is None 2021-02-20T08:04:56.421 DEBUG:teuthology.orchestra.run.smithi003:> sudo lsb_release -is 2021-02-20T08:04:56.477 INFO:teuthology.orchestra.run.smithi003.stdout:Ubuntu 2021-02-20T08:04:56.482 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-02-20T08:04:56.483 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=d318d42d502ea13c78f2af2499efd19dcc7e174b 2021-02-20T08:04:56.655 DEBUG:teuthology.orchestra.run.smithi003:> dpkg-query -W -f '${Version}' ceph 2021-02-20T08:04:56.680 INFO:teuthology.orchestra.run.smithi003.stdout:17.0.0-929-gd318d42d-1bionic 2021-02-20T08:04:56.681 INFO:teuthology.packaging:The installed version of ceph is 17.0.0-929-gd318d42d-1bionic 2021-02-20T08:04:56.681 INFO:teuthology.task.install:The correct ceph version 17.0.0-929-gd318d42d-1bionic is installed. 2021-02-20T08:04:56.682 DEBUG:teuthology.orchestra.run.smithi090:> sudo lsb_release -is 2021-02-20T08:04:56.736 INFO:teuthology.orchestra.run.smithi090.stdout:Ubuntu 2021-02-20T08:04:56.741 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-02-20T08:04:56.742 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=d318d42d502ea13c78f2af2499efd19dcc7e174b 2021-02-20T08:04:56.905 DEBUG:teuthology.orchestra.run.smithi090:> dpkg-query -W -f '${Version}' ceph 2021-02-20T08:04:56.929 INFO:teuthology.orchestra.run.smithi090.stdout:17.0.0-929-gd318d42d-1bionic 2021-02-20T08:04:56.930 INFO:teuthology.packaging:The installed version of ceph is 17.0.0-929-gd318d42d-1bionic 2021-02-20T08:04:56.930 INFO:teuthology.task.install:The correct ceph version 17.0.0-929-gd318d42d-1bionic is installed. 2021-02-20T08:04:56.931 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2021-02-20T08:04:56.933 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:04:56.933 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-02-20T08:04:56.949 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:04:56.949 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-02-20T08:04:56.985 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2021-02-20T08:04:56.987 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:04:56.988 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/daemon-helper 2021-02-20T08:04:57.005 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-02-20T08:04:57.059 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:04:57.059 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/daemon-helper 2021-02-20T08:04:57.074 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-02-20T08:04:57.133 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2021-02-20T08:04:57.133 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:04:57.134 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/adjust-ulimits 2021-02-20T08:04:57.149 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-02-20T08:04:57.205 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:04:57.205 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/adjust-ulimits 2021-02-20T08:04:57.219 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-02-20T08:04:57.277 INFO:teuthology.run_tasks:Running task ceph... 2021-02-20T08:04:57.359 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2021-02-20T08:04:57.360 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 777 /var/log/ceph 2021-02-20T08:04:57.363 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 777 /var/log/ceph 2021-02-20T08:04:57.378 INFO:tasks.ceph:Disabling ceph logrotate... 2021-02-20T08:04:57.378 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/logrotate.d/ceph 2021-02-20T08:04:57.420 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/logrotate.d/ceph 2021-02-20T08:04:57.434 INFO:tasks.ceph:Creating extra log directories... 2021-02-20T08:04:57.435 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-02-20T08:04:57.476 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-02-20T08:04:57.536 INFO:tasks.ceph:Creating ceph cluster cluster1... 2021-02-20T08:04:57.537 INFO:tasks.ceph:config {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'none'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(OSD_SLOW_PING_TIME'], 'cpu_profile': set(), 'cluster': 'cluster1', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2021-02-20T08:04:57.537 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi/5897918', 'branch': 'wip-mgolub-testing', 'description': 'rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}', 'email': 'to.my.trociny@gmail.com', 'first_in_suite': False, 'job_id': '5897918', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': 'run two ceph clusters and install rbd-mirror'}, {'desc': 'configure the permissions for client.mirror'}, {'desc': '2 ceph clusters with 1 mon and 3 osds each'}, {'desc': 'run four rbd-mirror daemons per cluster'}, {'desc': 'run the rbd_mirror_stress.sh workunit to test the rbd-mirror daemon'}], 'name': 'trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '18.04', 'overrides': {'admin_socket': {'branch': 'wip-mgolub-testing'}, 'ceph': {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'none'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(OSD_SLOW_PING_TIME'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mgolub-testing', 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}}, 'owner': 'scheduled_trociny@teuthology', 'priority': 50, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'], ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']], 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b', 'sleep_before_teardown': 0, 'suite': 'rbd', 'suite_branch': 'wip-mgolub-testing', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_d318d42d502ea13c78f2af2499efd19dcc7e174b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b', 'targets': {'smithi003.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyx++l8vCCc5LmHtTy8pQxD4u432ooKGE/AHMpohypjtB5vWw9eqB1m8w0iOU+wGGonfdzP49qcnKxw7NUtQh2vSkRA2kXw+sz6IzEsm7eqVV2UypIjsbl09hZB4m94uYSl5urqZ3IbVMVYUDtH+NAU8I+ghkhkTRSnb8Lc4bQmOiKDRyNXAa0mdofbiTcj32PSs8iWuDN2imSWHlUeBFjOvDKD1IGBic/qXPgTjWhDVTGyyT1GNM1Sv5pTbmpxLyxjm1qTMHVpU1nFgQTlUW4AJdmqF5ASBFlNJJjCxGguxlXAqqCbbz5cgzYnB1bwf7IzOtHBonnYp7ydVH0JDjB', 'smithi090.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJHRcfKBV/OkZP8o1/N7APaJg3UhsSRUYymD9kh3n2b7ZdoWr7TS7S8ZaUjR/zdYkdiX/JxbVf/WciS0KJ02FYSNpPTMHOgeqyw1Ih37MiOXKzwsg1PxdZmiqyNGkAlqr5pY61dA6xWSPgmvbvrmZlHkX97KC5EBl7gsvJokmA3YidM5VbGerWOLI/bRmbpzi8wJh9Ovm+rbln7Es0Jh7QBARXsFU9QNocjPWXhbHv7049UrfKUbAfiyBRpyFx6cePrOWl5P7lqO1h90JO8LxUa5c8ByjRGGE7PcMFIcQq0TyGAgSPnLdU4b6UM3z6FNPKL4pxeR1CuyiHjS5JF7XL'}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': ['rbd-mirror'], 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}}, {'ceph': {'cluster': 'cluster1', 'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'none'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(OSD_SLOW_PING_TIME'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}}, {'ceph': {'cluster': 'cluster2'}}, {'exec': {'cluster1.client.mirror.0': ["sudo ceph --cluster cluster1 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'"]}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.3', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.3', 'thrash': True}}, {'rbd-mirror-thrash': {'cluster': 'cluster1'}}, {'rbd-mirror-thrash': {'cluster': 'cluster2'}}, {'workunit': {'clients': {'cluster1.client.mirror': ['rbd/rbd_mirror_stress.sh']}, 'env': {'CEPH_ARGS': '', 'MIRROR_IMAGE_MODE': 'snapshot', 'MIRROR_POOL_MODE': 'image', 'RBD_IMAGE_FEATURES': 'layering,exclusive-lock,object-map,fast-diff', 'RBD_MIRROR_INSTANCES': '4', 'RBD_MIRROR_USE_EXISTING_CLUSTER': '1', 'RBD_MIRROR_USE_RBD_MIRROR': '1'}, 'timeout': '6h'}}], 'teuthology_branch': 'master', 'tube': 'smithi', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.29353'} 2021-02-20T08:04:57.538 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/cluster1.data 2021-02-20T08:04:57.541 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/cluster1.data 2021-02-20T08:04:57.548 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/run/ceph 2021-02-20T08:04:57.588 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m0777 -- /var/run/ceph 2021-02-20T08:04:57.605 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:04:57.605 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/scratch_devs of=/dev/stdout 2021-02-20T08:04:57.649 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-02-20T08:04:57.650 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-02-20T08:04:57.697 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-02-20T08:04:57.697 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T08:04:57.697 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 343 Links: 1 2021-02-20T08:04:57.697 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T08:04:57.698 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2021-02-20 08:02:15.853322830 +0000 2021-02-20T08:04:57.698 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2021-02-20 08:02:15.753322831 +0000 2021-02-20T08:04:57.698 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2021-02-20 08:02:15.753322831 +0000 2021-02-20T08:04:57.698 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2021-02-20T08:04:57.707 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2021-02-20T08:04:57.707 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2021-02-20T08:04:57.707 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000399804 s, 1.3 MB/s 2021-02-20T08:04:57.711 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-02-20T08:04:57.765 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-02-20T08:04:57.765 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T08:04:57.766 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 349 Links: 1 2021-02-20T08:04:57.766 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T08:04:57.766 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2021-02-20 08:02:16.081322827 +0000 2021-02-20T08:04:57.766 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2021-02-20 08:02:15.993322828 +0000 2021-02-20T08:04:57.767 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2021-02-20 08:02:15.993322828 +0000 2021-02-20T08:04:57.767 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2021-02-20T08:04:57.774 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2021-02-20T08:04:57.774 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2021-02-20T08:04:57.774 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000364427 s, 1.4 MB/s 2021-02-20T08:04:57.778 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-02-20T08:04:57.825 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-02-20T08:04:57.825 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T08:04:57.825 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 355 Links: 1 2021-02-20T08:04:57.826 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T08:04:57.826 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2021-02-20 08:02:16.317322825 +0000 2021-02-20T08:04:57.826 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2021-02-20 08:02:16.209322826 +0000 2021-02-20T08:04:57.826 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2021-02-20 08:02:16.209322826 +0000 2021-02-20T08:04:57.826 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2021-02-20T08:04:57.835 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2021-02-20T08:04:57.835 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2021-02-20T08:04:57.835 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000330731 s, 1.5 MB/s 2021-02-20T08:04:57.842 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-02-20T08:04:57.889 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-02-20T08:04:57.889 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T08:04:57.889 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 361 Links: 1 2021-02-20T08:04:57.890 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T08:04:57.890 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2021-02-20 08:02:16.445322823 +0000 2021-02-20T08:04:57.890 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2021-02-20 08:02:16.441322824 +0000 2021-02-20T08:04:57.891 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2021-02-20 08:02:16.441322824 +0000 2021-02-20T08:04:57.891 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2021-02-20T08:04:57.898 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2021-02-20T08:04:57.898 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2021-02-20T08:04:57.898 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000365285 s, 1.4 MB/s 2021-02-20T08:04:57.902 INFO:tasks.ceph:osd dev map: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T08:04:57.903 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}} 2021-02-20T08:04:57.903 INFO:tasks.ceph:Generating config... 2021-02-20T08:04:57.906 INFO:tasks.ceph:[client] debug rbd = 20 2021-02-20T08:04:57.907 INFO:tasks.ceph:[client] debug rbd_mirror = 15 2021-02-20T08:04:57.907 INFO:tasks.ceph:[client] log to stderr = False 2021-02-20T08:04:57.908 INFO:tasks.ceph:[client] rbd default features = 125 2021-02-20T08:04:57.908 INFO:tasks.ceph:[client] rbd mirror image policy type = none 2021-02-20T08:04:57.908 INFO:tasks.ceph:[client.mirror.0] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:04:57.909 INFO:tasks.ceph:[client.mirror.0] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:04:57.910 INFO:tasks.ceph:[client.mirror.1] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:04:57.910 INFO:tasks.ceph:[client.mirror.1] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:04:57.910 INFO:tasks.ceph:[client.mirror.2] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:04:57.911 INFO:tasks.ceph:[client.mirror.2] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:04:57.911 INFO:tasks.ceph:[client.mirror.3] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:04:57.911 INFO:tasks.ceph:[client.mirror.3] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:04:57.912 INFO:tasks.ceph:[global] mon client directed command retry = 5 2021-02-20T08:04:57.912 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2021-02-20T08:04:57.912 INFO:tasks.ceph:[mgr] debug mgr = 20 2021-02-20T08:04:57.913 INFO:tasks.ceph:[mgr] debug ms = 1 2021-02-20T08:04:57.913 INFO:tasks.ceph:[mon] debug mon = 20 2021-02-20T08:04:57.913 INFO:tasks.ceph:[mon] debug ms = 1 2021-02-20T08:04:57.914 INFO:tasks.ceph:[mon] debug paxos = 20 2021-02-20T08:04:57.914 INFO:tasks.ceph:[osd] bdev async discard = True 2021-02-20T08:04:57.914 INFO:tasks.ceph:[osd] bdev enable discard = True 2021-02-20T08:04:57.914 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2021-02-20T08:04:57.915 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2021-02-20T08:04:57.915 INFO:tasks.ceph:[osd] debug bluefs = 20 2021-02-20T08:04:57.915 INFO:tasks.ceph:[osd] debug bluestore = 20 2021-02-20T08:04:57.916 INFO:tasks.ceph:[osd] debug filestore = 20 2021-02-20T08:04:57.916 INFO:tasks.ceph:[osd] debug journal = 20 2021-02-20T08:04:57.916 INFO:tasks.ceph:[osd] debug ms = 20 2021-02-20T08:04:57.916 INFO:tasks.ceph:[osd] debug osd = 25 2021-02-20T08:04:57.917 INFO:tasks.ceph:[osd] debug rocksdb = 10 2021-02-20T08:04:57.917 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2021-02-20T08:04:57.917 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2021-02-20T08:04:57.918 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2021-02-20T08:04:57.918 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2021-02-20T08:04:57.918 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2021-02-20T08:04:57.918 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2021-02-20T08:04:57.919 INFO:tasks.ceph:Setting up cluster1.mon.a... 2021-02-20T08:04:57.920 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/cluster1.keyring 2021-02-20T08:04:57.978 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/cluster1.keyring 2021-02-20T08:04:57.980 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/cluster1.keyring 2021-02-20T08:04:58.058 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2021-02-20T08:04:58.115 DEBUG:tasks.ceph:Ceph mon addresses: [('cluster1.mon.a', '172.21.15.3')] 2021-02-20T08:04:58.116 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.3', 'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'none'}, 'mon.a': {}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}} 2021-02-20T08:04:58.117 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:04:58.117 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2021-02-20T08:04:58.164 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.3 --print /home/ubuntu/cephtest/cluster1.monmap 2021-02-20T08:04:58.237 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: monmap file /home/ubuntu/cephtest/cluster1.monmap 2021-02-20T08:04:58.237 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: generated fsid 5f712db1-9e20-4f92-8743-04e14bd4d26e 2021-02-20T08:04:58.238 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 0 2021-02-20T08:04:58.238 INFO:teuthology.orchestra.run.smithi003.stdout:fsid 5f712db1-9e20-4f92-8743-04e14bd4d26e 2021-02-20T08:04:58.238 INFO:teuthology.orchestra.run.smithi003.stdout:last_changed 2021-02-20T08:04:58.238305+0000 2021-02-20T08:04:58.238 INFO:teuthology.orchestra.run.smithi003.stdout:created 2021-02-20T08:04:58.238305+0000 2021-02-20T08:04:58.238 INFO:teuthology.orchestra.run.smithi003.stdout:min_mon_release 0 (unknown) 2021-02-20T08:04:58.239 INFO:teuthology.orchestra.run.smithi003.stdout:election_strategy: 1 2021-02-20T08:04:58.239 INFO:teuthology.orchestra.run.smithi003.stdout:0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2021-02-20T08:04:58.239 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/cluster1.monmap (1 monitors) 2021-02-20T08:04:58.240 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2021-02-20T08:04:58.288 INFO:tasks.ceph:Writing /etc/ceph/cluster1.conf for FSID 5f712db1-9e20-4f92-8743-04e14bd4d26e... 2021-02-20T08:04:58.289 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster1.conf && sudo chmod 0644 /etc/ceph/cluster1.conf > /dev/null 2021-02-20T08:04:58.332 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster1.conf && sudo chmod 0644 /etc/ceph/cluster1.conf > /dev/null 2021-02-20T08:04:58.364 INFO:teuthology.orchestra.run.smithi090.stdout:[global] 2021-02-20T08:04:58.365 INFO:teuthology.orchestra.run.smithi090.stdout: chdir = "" 2021-02-20T08:04:58.365 INFO:teuthology.orchestra.run.smithi090.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-20T08:04:58.365 INFO:teuthology.orchestra.run.smithi090.stdout: auth supported = cephx 2021-02-20T08:04:58.365 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.365 INFO:teuthology.orchestra.run.smithi090.stdout: filestore xattr use omap = true 2021-02-20T08:04:58.366 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.366 INFO:teuthology.orchestra.run.smithi090.stdout: mon clock drift allowed = 1.000 2021-02-20T08:04:58.366 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.366 INFO:teuthology.orchestra.run.smithi090.stdout: osd crush chooseleaf type = 0 2021-02-20T08:04:58.367 INFO:teuthology.orchestra.run.smithi090.stdout: auth debug = true 2021-02-20T08:04:58.367 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.367 INFO:teuthology.orchestra.run.smithi090.stdout: ms die on old message = true 2021-02-20T08:04:58.367 INFO:teuthology.orchestra.run.smithi090.stdout: ms die on bug = true 2021-02-20T08:04:58.367 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.368 INFO:teuthology.orchestra.run.smithi090.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-20T08:04:58.368 INFO:teuthology.orchestra.run.smithi090.stdout: mon pg warn max object skew = 0 2021-02-20T08:04:58.368 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.368 INFO:teuthology.orchestra.run.smithi090.stdout: # disable pg_autoscaler by default for new pools 2021-02-20T08:04:58.368 INFO:teuthology.orchestra.run.smithi090.stdout: osd_pool_default_pg_autoscale_mode = off 2021-02-20T08:04:58.369 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.369 INFO:teuthology.orchestra.run.smithi090.stdout: osd pool default size = 2 2021-02-20T08:04:58.369 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.369 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd allow primary affinity = true 2021-02-20T08:04:58.369 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd allow pg remap = true 2021-02-20T08:04:58.370 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on legacy crush tunables = false 2021-02-20T08:04:58.370 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on crush straw calc version zero = false 2021-02-20T08:04:58.370 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on no sortbitwise = false 2021-02-20T08:04:58.370 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on osd down out interval zero = false 2021-02-20T08:04:58.371 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on too few osds = false 2021-02-20T08:04:58.371 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-20T08:04:58.371 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-20T08:04:58.371 INFO:teuthology.orchestra.run.smithi090.stdout: mon_allow_pool_size_one = true 2021-02-20T08:04:58.371 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.372 INFO:teuthology.orchestra.run.smithi090.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-20T08:04:58.372 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.372 INFO:teuthology.orchestra.run.smithi090.stdout: osd default data pool replay window = 5 2021-02-20T08:04:58.372 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.372 INFO:teuthology.orchestra.run.smithi090.stdout: mon allow pool delete = true 2021-02-20T08:04:58.373 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.373 INFO:teuthology.orchestra.run.smithi090.stdout: mon cluster log file level = debug 2021-02-20T08:04:58.373 INFO:teuthology.orchestra.run.smithi090.stdout: debug asserts on shutdown = true 2021-02-20T08:04:58.373 INFO:teuthology.orchestra.run.smithi090.stdout: mon health detail to clog = false 2021-02-20T08:04:58.373 INFO:teuthology.orchestra.run.smithi090.stdout: mon host = 172.21.15.3 2021-02-20T08:04:58.374 INFO:teuthology.orchestra.run.smithi090.stdout: mon client directed command retry = 5 2021-02-20T08:04:58.374 INFO:teuthology.orchestra.run.smithi090.stdout: ms inject socket failures = 5000 2021-02-20T08:04:58.374 INFO:teuthology.orchestra.run.smithi090.stdout: fsid = 5f712db1-9e20-4f92-8743-04e14bd4d26e 2021-02-20T08:04:58.374 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.375 INFO:teuthology.orchestra.run.smithi090.stdout:[osd] 2021-02-20T08:04:58.375 INFO:teuthology.orchestra.run.smithi090.stdout: osd journal size = 100 2021-02-20T08:04:58.375 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.375 INFO:teuthology.orchestra.run.smithi090.stdout: osd scrub load threshold = 5.0 2021-02-20T08:04:58.376 INFO:teuthology.orchestra.run.smithi090.stdout: osd scrub max interval = 600 2021-02-20T08:04:58.376 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.376 INFO:teuthology.orchestra.run.smithi090.stdout: osd recover clone overlap = true 2021-02-20T08:04:58.376 INFO:teuthology.orchestra.run.smithi090.stdout: osd recovery max chunk = 1048576 2021-02-20T08:04:58.376 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.377 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug shutdown = true 2021-02-20T08:04:58.377 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug op order = true 2021-02-20T08:04:58.377 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug verify stray on activate = true 2021-02-20T08:04:58.377 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.377 INFO:teuthology.orchestra.run.smithi090.stdout: osd open classes on start = true 2021-02-20T08:04:58.378 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug pg log writeout = true 2021-02-20T08:04:58.378 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.378 INFO:teuthology.orchestra.run.smithi090.stdout: osd deep scrub update digest min age = 30 2021-02-20T08:04:58.378 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.378 INFO:teuthology.orchestra.run.smithi090.stdout: osd map max advance = 10 2021-02-20T08:04:58.379 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.379 INFO:teuthology.orchestra.run.smithi090.stdout: journal zero on create = true 2021-02-20T08:04:58.379 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.379 INFO:teuthology.orchestra.run.smithi090.stdout: filestore ondisk finisher threads = 3 2021-02-20T08:04:58.379 INFO:teuthology.orchestra.run.smithi090.stdout: filestore apply finisher threads = 3 2021-02-20T08:04:58.380 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.380 INFO:teuthology.orchestra.run.smithi090.stdout: bdev debug aio = true 2021-02-20T08:04:58.380 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug misdirected ops = true 2021-02-20T08:04:58.380 INFO:teuthology.orchestra.run.smithi090.stdout: bdev async discard = True 2021-02-20T08:04:58.380 INFO:teuthology.orchestra.run.smithi090.stdout: bdev enable discard = True 2021-02-20T08:04:58.381 INFO:teuthology.orchestra.run.smithi090.stdout: bluestore block size = 96636764160 2021-02-20T08:04:58.381 INFO:teuthology.orchestra.run.smithi090.stdout: bluestore fsck on mount = True 2021-02-20T08:04:58.381 INFO:teuthology.orchestra.run.smithi090.stdout: debug bluefs = 20 2021-02-20T08:04:58.381 INFO:teuthology.orchestra.run.smithi090.stdout: debug bluestore = 20 2021-02-20T08:04:58.382 INFO:teuthology.orchestra.run.smithi090.stdout: debug filestore = 20 2021-02-20T08:04:58.382 INFO:teuthology.orchestra.run.smithi090.stdout: debug journal = 20 2021-02-20T08:04:58.382 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 20 2021-02-20T08:04:58.382 INFO:teuthology.orchestra.run.smithi090.stdout: debug osd = 25 2021-02-20T08:04:58.382 INFO:teuthology.orchestra.run.smithi090.stdout: debug rocksdb = 10 2021-02-20T08:04:58.383 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd backfillfull_ratio = 0.85 2021-02-20T08:04:58.383 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd full ratio = 0.9 2021-02-20T08:04:58.383 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd nearfull ratio = 0.8 2021-02-20T08:04:58.383 INFO:teuthology.orchestra.run.smithi090.stdout: osd failsafe full ratio = 0.95 2021-02-20T08:04:58.383 INFO:teuthology.orchestra.run.smithi090.stdout: osd memory target = 1610612736 2021-02-20T08:04:58.384 INFO:teuthology.orchestra.run.smithi090.stdout: osd objectstore = bluestore 2021-02-20T08:04:58.384 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.384 INFO:teuthology.orchestra.run.smithi090.stdout:[mgr] 2021-02-20T08:04:58.384 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2021-02-20T08:04:58.385 INFO:teuthology.orchestra.run.smithi090.stdout: debug mgr = 20 2021-02-20T08:04:58.385 INFO:teuthology.orchestra.run.smithi090.stdout: debug mon = 20 2021-02-20T08:04:58.385 INFO:teuthology.orchestra.run.smithi090.stdout: debug auth = 20 2021-02-20T08:04:58.385 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min pgs per osd = 4 2021-02-20T08:04:58.385 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min bytes per osd = 10 2021-02-20T08:04:58.386 INFO:teuthology.orchestra.run.smithi090.stdout: mgr/telemetry/nag = false 2021-02-20T08:04:58.386 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.386 INFO:teuthology.orchestra.run.smithi090.stdout:[mon] 2021-02-20T08:04:58.386 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2021-02-20T08:04:58.386 INFO:teuthology.orchestra.run.smithi090.stdout: debug mon = 20 2021-02-20T08:04:58.387 INFO:teuthology.orchestra.run.smithi090.stdout: debug paxos = 20 2021-02-20T08:04:58.387 INFO:teuthology.orchestra.run.smithi090.stdout: debug auth = 20 2021-02-20T08:04:58.387 INFO:teuthology.orchestra.run.smithi090.stdout: mon data avail warn = 5 2021-02-20T08:04:58.387 INFO:teuthology.orchestra.run.smithi090.stdout: mon mgr mkfs grace = 240 2021-02-20T08:04:58.388 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min pgs per osd = 4 2021-02-20T08:04:58.389 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd reporter subtree level = osd 2021-02-20T08:04:58.389 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd prime pg temp = true 2021-02-20T08:04:58.389 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min bytes per osd = 10 2021-02-20T08:04:58.389 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:04:58.390 INFO:teuthology.orchestra.run.smithi090.stdout:[client] 2021-02-20T08:04:58.390 INFO:teuthology.orchestra.run.smithi090.stdout: rgw cache enabled = true 2021-02-20T08:04:58.391 INFO:teuthology.orchestra.run.smithi090.stdout: rgw enable ops log = true 2021-02-20T08:04:58.391 INFO:teuthology.orchestra.run.smithi090.stdout: rgw enable usage log = true 2021-02-20T08:04:58.391 INFO:teuthology.orchestra.run.smithi090.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-20T08:04:58.392 INFO:teuthology.orchestra.run.smithi090.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-20T08:04:58.392 INFO:teuthology.orchestra.run.smithi090.stdout: debug rbd = 20 2021-02-20T08:04:58.393 INFO:teuthology.orchestra.run.smithi090.stdout: debug rbd_mirror = 15 2021-02-20T08:04:58.393 INFO:teuthology.orchestra.run.smithi090.stdout: log to stderr = False 2021-02-20T08:04:58.393 INFO:teuthology.orchestra.run.smithi090.stdout: rbd default features = 125 2021-02-20T08:04:58.393 INFO:teuthology.orchestra.run.smithi090.stdout: rbd mirror image policy type = none 2021-02-20T08:04:58.393 INFO:teuthology.orchestra.run.smithi090.stdout:[mon.a] 2021-02-20T08:04:58.394 INFO:teuthology.orchestra.run.smithi090.stdout:[client.mirror.0] 2021-02-20T08:04:58.394 INFO:teuthology.orchestra.run.smithi090.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:04:58.395 INFO:teuthology.orchestra.run.smithi090.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:04:58.395 INFO:teuthology.orchestra.run.smithi090.stdout:[client.mirror.1] 2021-02-20T08:04:58.395 INFO:teuthology.orchestra.run.smithi090.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:04:58.397 INFO:teuthology.orchestra.run.smithi090.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:04:58.397 INFO:teuthology.orchestra.run.smithi090.stdout:[client.mirror.2] 2021-02-20T08:04:58.398 INFO:teuthology.orchestra.run.smithi090.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:04:58.398 INFO:teuthology.orchestra.run.smithi090.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:04:58.399 INFO:teuthology.orchestra.run.smithi090.stdout:[client.mirror.3] 2021-02-20T08:04:58.399 INFO:teuthology.orchestra.run.smithi090.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:04:58.399 INFO:teuthology.orchestra.run.smithi090.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:04:58.401 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2021-02-20T08:04:58.401 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2021-02-20T08:04:58.402 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-20T08:04:58.404 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2021-02-20T08:04:58.404 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2021-02-20T08:04:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2021-02-20T08:04:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.406 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2021-02-20T08:04:58.406 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2021-02-20T08:04:58.406 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.406 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2021-02-20T08:04:58.407 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2021-02-20T08:04:58.407 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.407 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-20T08:04:58.407 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2021-02-20T08:04:58.407 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.408 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2021-02-20T08:04:58.408 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2021-02-20T08:04:58.408 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.408 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2021-02-20T08:04:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2021-02-20T08:04:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2021-02-20T08:04:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2021-02-20T08:04:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2021-02-20T08:04:58.410 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2021-02-20T08:04:58.410 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2021-02-20T08:04:58.410 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2021-02-20T08:04:58.410 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-20T08:04:58.410 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-20T08:04:58.411 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2021-02-20T08:04:58.411 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.411 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-20T08:04:58.411 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2021-02-20T08:04:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2021-02-20T08:04:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2021-02-20T08:04:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2021-02-20T08:04:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2021-02-20T08:04:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = 172.21.15.3 2021-02-20T08:04:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: mon client directed command retry = 5 2021-02-20T08:04:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: ms inject socket failures = 5000 2021-02-20T08:04:58.414 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 5f712db1-9e20-4f92-8743-04e14bd4d26e 2021-02-20T08:04:58.414 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.414 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2021-02-20T08:04:58.414 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2021-02-20T08:04:58.414 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.415 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2021-02-20T08:04:58.415 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2021-02-20T08:04:58.415 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.415 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2021-02-20T08:04:58.416 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2021-02-20T08:04:58.416 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.416 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2021-02-20T08:04:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2021-02-20T08:04:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2021-02-20T08:04:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2021-02-20T08:04:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2021-02-20T08:04:58.418 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.418 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2021-02-20T08:04:58.418 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.418 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2021-02-20T08:04:58.418 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.419 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2021-02-20T08:04:58.419 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.419 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2021-02-20T08:04:58.419 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2021-02-20T08:04:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2021-02-20T08:04:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2021-02-20T08:04:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: bdev async discard = True 2021-02-20T08:04:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: bdev enable discard = True 2021-02-20T08:04:58.421 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2021-02-20T08:04:58.421 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2021-02-20T08:04:58.421 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 20 2021-02-20T08:04:58.421 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 20 2021-02-20T08:04:58.421 INFO:teuthology.orchestra.run.smithi003.stdout: debug filestore = 20 2021-02-20T08:04:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: debug journal = 20 2021-02-20T08:04:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 20 2021-02-20T08:04:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 25 2021-02-20T08:04:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 10 2021-02-20T08:04:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2021-02-20T08:04:58.423 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2021-02-20T08:04:58.423 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2021-02-20T08:04:58.423 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2021-02-20T08:04:58.423 INFO:teuthology.orchestra.run.smithi003.stdout: osd memory target = 1610612736 2021-02-20T08:04:58.423 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2021-02-20T08:04:58.424 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.424 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2021-02-20T08:04:58.424 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2021-02-20T08:04:58.424 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2021-02-20T08:04:58.425 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2021-02-20T08:04:58.425 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2021-02-20T08:04:58.425 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2021-02-20T08:04:58.425 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2021-02-20T08:04:58.425 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2021-02-20T08:04:58.426 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.426 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2021-02-20T08:04:58.426 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2021-02-20T08:04:58.426 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2021-02-20T08:04:58.426 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2021-02-20T08:04:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2021-02-20T08:04:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2021-02-20T08:04:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2021-02-20T08:04:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2021-02-20T08:04:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2021-02-20T08:04:58.428 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2021-02-20T08:04:58.428 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2021-02-20T08:04:58.428 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:04:58.428 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2021-02-20T08:04:58.428 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2021-02-20T08:04:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2021-02-20T08:04:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2021-02-20T08:04:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-20T08:04:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-20T08:04:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: debug rbd = 20 2021-02-20T08:04:58.430 INFO:teuthology.orchestra.run.smithi003.stdout: debug rbd_mirror = 15 2021-02-20T08:04:58.430 INFO:teuthology.orchestra.run.smithi003.stdout: log to stderr = False 2021-02-20T08:04:58.430 INFO:teuthology.orchestra.run.smithi003.stdout: rbd default features = 125 2021-02-20T08:04:58.430 INFO:teuthology.orchestra.run.smithi003.stdout: rbd mirror image policy type = none 2021-02-20T08:04:58.431 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2021-02-20T08:04:58.431 INFO:teuthology.orchestra.run.smithi003.stdout:[client.mirror.0] 2021-02-20T08:04:58.431 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:04:58.431 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:04:58.431 INFO:teuthology.orchestra.run.smithi003.stdout:[client.mirror.1] 2021-02-20T08:04:58.432 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:04:58.432 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:04:58.432 INFO:teuthology.orchestra.run.smithi003.stdout:[client.mirror.2] 2021-02-20T08:04:58.432 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:04:58.432 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:04:58.433 INFO:teuthology.orchestra.run.smithi003.stdout:[client.mirror.3] 2021-02-20T08:04:58.433 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:04:58.433 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:04:58.433 INFO:tasks.ceph:Creating admin key on cluster1.mon.a... 2021-02-20T08:04:58.434 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/cluster1.keyring 2021-02-20T08:04:58.488 INFO:tasks.ceph:Copying monmap to all nodes... 2021-02-20T08:04:58.488 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:04:58.489 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/cluster1.keyring of=/dev/stdout 2021-02-20T08:04:58.542 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:04:58.543 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/cluster1.monmap of=/dev/stdout 2021-02-20T08:04:58.592 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi003.front.sepia.ceph.com 2021-02-20T08:04:58.592 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:04:58.593 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/cluster1.keyring 2021-02-20T08:04:58.593 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2021-02-20T08:04:58.659 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:04:58.659 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/cluster1.monmap 2021-02-20T08:04:58.708 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi090.front.sepia.ceph.com 2021-02-20T08:04:58.709 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:04:58.710 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/cluster1.keyring 2021-02-20T08:04:58.710 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2021-02-20T08:04:58.735 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:04:58.736 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/home/ubuntu/cephtest/cluster1.monmap 2021-02-20T08:04:58.785 INFO:tasks.ceph:Setting up mon nodes... 2021-02-20T08:04:58.785 INFO:tasks.ceph:Setting up mgr nodes... 2021-02-20T08:04:58.786 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mgr/cluster1-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/cluster1-x/keyring 2021-02-20T08:04:58.830 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mgr/cluster1-x/keyring 2021-02-20T08:04:58.832 INFO:tasks.ceph:Setting up mds nodes... 2021-02-20T08:04:58.832 INFO:tasks.ceph_client:Setting up client nodes... 2021-02-20T08:04:58.833 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/cluster1.client.0.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.0.keyring 2021-02-20T08:04:58.908 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/cluster1.client.0.keyring 2021-02-20T08:04:58.919 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror /etc/ceph/cluster1.client.mirror.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.keyring 2021-02-20T08:04:58.952 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/cluster1.client.mirror.keyring 2021-02-20T08:04:58.965 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.0 /etc/ceph/cluster1.client.mirror.0.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.0.keyring 2021-02-20T08:04:59.040 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/cluster1.client.mirror.0.keyring 2021-02-20T08:04:59.050 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.1 /etc/ceph/cluster1.client.mirror.1.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.1.keyring 2021-02-20T08:04:59.127 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/cluster1.client.mirror.1.keyring 2021-02-20T08:04:59.139 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.2 /etc/ceph/cluster1.client.mirror.2.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.2.keyring 2021-02-20T08:04:59.217 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/cluster1.client.mirror.2.keyring 2021-02-20T08:04:59.229 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.3 /etc/ceph/cluster1.client.mirror.3.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.3.keyring 2021-02-20T08:04:59.303 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/cluster1.client.mirror.3.keyring 2021-02-20T08:04:59.315 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.4 /etc/ceph/cluster1.client.mirror.4.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.4.keyring 2021-02-20T08:04:59.391 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/cluster1.client.mirror.4.keyring 2021-02-20T08:04:59.403 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.5 /etc/ceph/cluster1.client.mirror.5.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.5.keyring 2021-02-20T08:04:59.479 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/cluster1.client.mirror.5.keyring 2021-02-20T08:04:59.491 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.6 /etc/ceph/cluster1.client.mirror.6.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.6.keyring 2021-02-20T08:04:59.567 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/cluster1.client.mirror.6.keyring 2021-02-20T08:04:59.579 INFO:tasks.ceph:Running mkfs on osd nodes... 2021-02-20T08:04:59.580 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}} 2021-02-20T08:04:59.580 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/cluster1-0 2021-02-20T08:04:59.596 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T08:04:59.597 INFO:tasks.ceph:role: cluster1.osd.0 2021-02-20T08:04:59.597 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com 2021-02-20T08:04:59.597 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-02-20T08:05:00.114 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2021-02-20T08:05:00.115 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-20T08:05:00.115 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-02-20T08:05:00.115 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-02-20T08:05:00.115 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2021-02-20T08:05:00.116 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-20T08:05:00.116 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-02-20T08:05:00.116 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-20T08:05:00.116 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-20T08:05:00.117 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2021-02-20T08:05:00.118 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/cluster1-0 2021-02-20T08:05:00.195 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-0 2021-02-20T08:05:00.207 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: /sbin/restorecon: command not found 2021-02-20T08:05:00.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:05:00.208 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/cluster1-1 2021-02-20T08:05:00.264 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T08:05:00.265 INFO:tasks.ceph:role: cluster1.osd.1 2021-02-20T08:05:00.265 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi003.front.sepia.ceph.com 2021-02-20T08:05:00.266 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-02-20T08:05:00.777 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2021-02-20T08:05:00.778 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-20T08:05:00.778 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-02-20T08:05:00.779 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-02-20T08:05:00.779 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2021-02-20T08:05:00.779 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-20T08:05:00.779 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-02-20T08:05:00.780 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-20T08:05:00.780 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-20T08:05:00.782 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2021-02-20T08:05:00.782 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/cluster1-1 2021-02-20T08:05:00.851 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-1 2021-02-20T08:05:00.861 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: /sbin/restorecon: command not found 2021-02-20T08:05:00.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:05:00.863 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/cluster1-2 2021-02-20T08:05:00.922 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T08:05:00.923 INFO:tasks.ceph:role: cluster1.osd.2 2021-02-20T08:05:00.923 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi003.front.sepia.ceph.com 2021-02-20T08:05:00.924 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-02-20T08:05:01.425 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2021-02-20T08:05:01.425 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-20T08:05:01.426 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-02-20T08:05:01.426 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-02-20T08:05:01.426 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2021-02-20T08:05:01.426 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-20T08:05:01.427 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-02-20T08:05:01.427 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-20T08:05:01.427 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-20T08:05:01.428 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2021-02-20T08:05:01.428 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/cluster1-2 2021-02-20T08:05:01.479 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-2 2021-02-20T08:05:01.489 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: /sbin/restorecon: command not found 2021-02-20T08:05:01.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:05:01.490 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/cluster1.monmap 2021-02-20T08:05:01.575 INFO:teuthology.orchestra.run.smithi003.stderr:2021-02-20T08:05:01.573+0000 7f9f9cd48080 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-0/keyring: can't open /var/lib/ceph/osd/cluster1-0/keyring: (2) No such file or directory 2021-02-20T08:05:01.578 INFO:teuthology.orchestra.run.smithi003.stderr:2021-02-20T08:05:01.573+0000 7f9f9cd48080 -1 created new key in keyring /var/lib/ceph/osd/cluster1-0/keyring 2021-02-20T08:05:01.578 INFO:teuthology.orchestra.run.smithi003.stderr:2021-02-20T08:05:01.573+0000 7f9f9cd48080 -1 bluestore(/var/lib/ceph/osd/cluster1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-0/block: (2) No such file or directory 2021-02-20T08:05:01.578 INFO:teuthology.orchestra.run.smithi003.stderr:2021-02-20T08:05:01.573+0000 7f9f9cd48080 -1 bluestore(/var/lib/ceph/osd/cluster1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-0/block: (2) No such file or directory 2021-02-20T08:05:01.579 INFO:teuthology.orchestra.run.smithi003.stderr:2021-02-20T08:05:01.573+0000 7f9f9cd48080 -1 bluestore(/var/lib/ceph/osd/cluster1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-0/block: (2) No such file or directory 2021-02-20T08:05:01.580 INFO:teuthology.orchestra.run.smithi003.stderr:2021-02-20T08:05:01.577+0000 7f9f9cd48080 -1 bluestore(/var/lib/ceph/osd/cluster1-0) _read_fsid unparsable uuid 2021-02-20T08:05:04.543 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-0 2021-02-20T08:05:04.560 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/cluster1.monmap 2021-02-20T08:05:04.641 INFO:teuthology.orchestra.run.smithi003.stderr:2021-02-20T08:05:04.637+0000 7fa02920d080 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-1/keyring: can't open /var/lib/ceph/osd/cluster1-1/keyring: (2) No such file or directory 2021-02-20T08:05:04.642 INFO:teuthology.orchestra.run.smithi003.stderr:2021-02-20T08:05:04.637+0000 7fa02920d080 -1 created new key in keyring /var/lib/ceph/osd/cluster1-1/keyring 2021-02-20T08:05:04.642 INFO:teuthology.orchestra.run.smithi003.stderr:2021-02-20T08:05:04.637+0000 7fa02920d080 -1 bluestore(/var/lib/ceph/osd/cluster1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-1/block: (2) No such file or directory 2021-02-20T08:05:04.642 INFO:teuthology.orchestra.run.smithi003.stderr:2021-02-20T08:05:04.637+0000 7fa02920d080 -1 bluestore(/var/lib/ceph/osd/cluster1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-1/block: (2) No such file or directory 2021-02-20T08:05:04.642 INFO:teuthology.orchestra.run.smithi003.stderr:2021-02-20T08:05:04.637+0000 7fa02920d080 -1 bluestore(/var/lib/ceph/osd/cluster1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-1/block: (2) No such file or directory 2021-02-20T08:05:04.643 INFO:teuthology.orchestra.run.smithi003.stderr:2021-02-20T08:05:04.641+0000 7fa02920d080 -1 bluestore(/var/lib/ceph/osd/cluster1-1) _read_fsid unparsable uuid 2021-02-20T08:05:07.599 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-1 2021-02-20T08:05:07.659 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/cluster1.monmap 2021-02-20T08:05:07.741 INFO:teuthology.orchestra.run.smithi003.stderr:2021-02-20T08:05:07.737+0000 7fcd96430080 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-2/keyring: can't open /var/lib/ceph/osd/cluster1-2/keyring: (2) No such file or directory 2021-02-20T08:05:07.742 INFO:teuthology.orchestra.run.smithi003.stderr:2021-02-20T08:05:07.737+0000 7fcd96430080 -1 created new key in keyring /var/lib/ceph/osd/cluster1-2/keyring 2021-02-20T08:05:07.742 INFO:teuthology.orchestra.run.smithi003.stderr:2021-02-20T08:05:07.737+0000 7fcd96430080 -1 bluestore(/var/lib/ceph/osd/cluster1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-2/block: (2) No such file or directory 2021-02-20T08:05:07.742 INFO:teuthology.orchestra.run.smithi003.stderr:2021-02-20T08:05:07.737+0000 7fcd96430080 -1 bluestore(/var/lib/ceph/osd/cluster1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-2/block: (2) No such file or directory 2021-02-20T08:05:07.742 INFO:teuthology.orchestra.run.smithi003.stderr:2021-02-20T08:05:07.737+0000 7fcd96430080 -1 bluestore(/var/lib/ceph/osd/cluster1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-2/block: (2) No such file or directory 2021-02-20T08:05:07.743 INFO:teuthology.orchestra.run.smithi003.stderr:2021-02-20T08:05:07.737+0000 7fcd96430080 -1 bluestore(/var/lib/ceph/osd/cluster1-2) _read_fsid unparsable uuid 2021-02-20T08:05:10.702 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-2 2021-02-20T08:05:10.757 INFO:tasks.ceph:Reading keys from all nodes... 2021-02-20T08:05:10.758 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:05:10.758 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mgr/cluster1-x/keyring of=/dev/stdout 2021-02-20T08:05:10.813 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:05:10.813 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/cluster1-0/keyring of=/dev/stdout 2021-02-20T08:05:10.867 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:05:10.868 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/cluster1-1/keyring of=/dev/stdout 2021-02-20T08:05:10.925 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:05:10.925 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/cluster1-2/keyring of=/dev/stdout 2021-02-20T08:05:10.980 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:05:10.981 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/cluster1.client.0.keyring of=/dev/stdout 2021-02-20T08:05:11.028 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:11.029 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/cluster1.client.mirror.keyring of=/dev/stdout 2021-02-20T08:05:11.037 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:11.038 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/cluster1.client.mirror.0.keyring of=/dev/stdout 2021-02-20T08:05:11.084 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:11.084 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/cluster1.client.mirror.1.keyring of=/dev/stdout 2021-02-20T08:05:11.133 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:11.134 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/cluster1.client.mirror.2.keyring of=/dev/stdout 2021-02-20T08:05:11.180 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:11.180 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/cluster1.client.mirror.3.keyring of=/dev/stdout 2021-02-20T08:05:11.229 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:11.229 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/cluster1.client.mirror.4.keyring of=/dev/stdout 2021-02-20T08:05:11.276 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:11.276 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/cluster1.client.mirror.5.keyring of=/dev/stdout 2021-02-20T08:05:11.324 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:11.325 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/cluster1.client.mirror.6.keyring of=/dev/stdout 2021-02-20T08:05:11.372 INFO:tasks.ceph:Adding keys to all mons... 2021-02-20T08:05:11.373 DEBUG:teuthology.orchestra.run.smithi003:> sudo tee -a /etc/ceph/cluster1.keyring 2021-02-20T08:05:11.386 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr.x] 2021-02-20T08:05:11.387 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCqwjBgGNOHMRAA6miss0xP1JOFPmFRNWuO4g== 2021-02-20T08:05:11.387 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.0] 2021-02-20T08:05:11.387 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCtwjBgB8VeIhAAKsTfwE809Phdgr5SrzEjAg== 2021-02-20T08:05:11.387 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.1] 2021-02-20T08:05:11.388 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCwwjBgGZFIJhAAU354uTm+AoNlmTN6C/oALw== 2021-02-20T08:05:11.388 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.2] 2021-02-20T08:05:11.388 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCzwjBgSDo8LBAAqQTydW4Sj/LjhAyuRHxocg== 2021-02-20T08:05:11.388 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2021-02-20T08:05:11.389 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCqwjBgemYwNhAAatVot26Vb/sX43STTcTpRA== 2021-02-20T08:05:11.389 INFO:teuthology.orchestra.run.smithi003.stdout:[client.mirror] 2021-02-20T08:05:11.389 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCqwjBgafngOBAAw3Ys0W9rqSUwcL5pC7kYLg== 2021-02-20T08:05:11.389 INFO:teuthology.orchestra.run.smithi003.stdout:[client.mirror.0] 2021-02-20T08:05:11.389 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCrwjBgeyJ7AhAA7fwgEFDU3mc7hQPkRwH2fw== 2021-02-20T08:05:11.390 INFO:teuthology.orchestra.run.smithi003.stdout:[client.mirror.1] 2021-02-20T08:05:11.390 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCrwjBgvhGyBxAAeeGE9qk38WS0DB4taWLEWQ== 2021-02-20T08:05:11.390 INFO:teuthology.orchestra.run.smithi003.stdout:[client.mirror.2] 2021-02-20T08:05:11.390 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCrwjBgjXILDRAA/udhZZB3FR9OL1ffD6yKeA== 2021-02-20T08:05:11.391 INFO:teuthology.orchestra.run.smithi003.stdout:[client.mirror.3] 2021-02-20T08:05:11.391 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCrwjBgcvArEhAAUrr61thh4umY2p4bjA2uUQ== 2021-02-20T08:05:11.391 INFO:teuthology.orchestra.run.smithi003.stdout:[client.mirror.4] 2021-02-20T08:05:11.391 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCrwjBgVMtuFxAAMC+P7JpyDZBFKMn62UtwDw== 2021-02-20T08:05:11.391 INFO:teuthology.orchestra.run.smithi003.stdout:[client.mirror.5] 2021-02-20T08:05:11.392 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCrwjBgmZ2oHBAAT+i0e/XXSynBMkZ5UCvluA== 2021-02-20T08:05:11.392 INFO:teuthology.orchestra.run.smithi003.stdout:[client.mirror.6] 2021-02-20T08:05:11.392 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCrwjBglSPsIRAA092dHFzGZcoirR8uCDMTRw== 2021-02-20T08:05:11.393 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-02-20T08:05:11.477 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-20T08:05:11.563 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-20T08:05:11.646 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-20T08:05:11.732 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T08:05:11.820 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T08:05:11.865 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T08:05:11.952 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T08:05:11.998 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T08:05:12.088 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T08:05:12.133 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.4 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T08:05:12.219 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.5 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T08:05:12.260 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.6 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T08:05:12.349 INFO:tasks.ceph:Running mkfs on mon nodes... 2021-02-20T08:05:12.350 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mon/cluster1-a 2021-02-20T08:05:12.360 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster cluster1 --mkfs -i a --monmap /home/ubuntu/cephtest/cluster1.monmap --keyring /etc/ceph/cluster1.keyring 2021-02-20T08:05:12.471 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mon/cluster1-a 2021-02-20T08:05:12.488 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/cluster1.monmap 2021-02-20T08:05:12.536 INFO:tasks.ceph:Starting mon daemons in cluster cluster1... 2021-02-20T08:05:12.536 INFO:tasks.ceph.cluster1.mon.a:Restarting daemon 2021-02-20T08:05:12.539 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster cluster1 -i a 2021-02-20T08:05:12.580 INFO:tasks.ceph.cluster1.mon.a:Started 2021-02-20T08:05:12.581 INFO:tasks.ceph:Starting mgr daemons in cluster cluster1... 2021-02-20T08:05:12.581 INFO:tasks.ceph.cluster1.mgr.x:Restarting daemon 2021-02-20T08:05:12.581 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster cluster1 -i x 2021-02-20T08:05:12.584 INFO:tasks.ceph.cluster1.mgr.x:Started 2021-02-20T08:05:12.584 INFO:tasks.ceph:Setting crush tunables to default 2021-02-20T08:05:12.585 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster cluster1 osd crush tunables default 2021-02-20T08:05:13.021 INFO:teuthology.orchestra.run.smithi003.stderr:adjusted tunables profile to default 2021-02-20T08:05:13.033 INFO:tasks.ceph:Starting osd daemons in cluster cluster1... 2021-02-20T08:05:13.033 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:05:13.034 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/cluster1-0/fsid of=/dev/stdout 2021-02-20T08:05:13.043 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:05:13.044 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/cluster1-1/fsid of=/dev/stdout 2021-02-20T08:05:13.098 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:05:13.099 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/cluster1-2/fsid of=/dev/stdout 2021-02-20T08:05:13.156 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster cluster1 osd new 530535a2-f247-4671-9d81-66b354c8f8c1 0 2021-02-20T08:05:13.420 INFO:teuthology.orchestra.run.smithi003.stdout:0 2021-02-20T08:05:13.432 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster cluster1 osd new d07084ae-a43e-4df0-b4d4-616b0e830ba4 1 2021-02-20T08:05:13.652 INFO:teuthology.orchestra.run.smithi003.stdout:1 2021-02-20T08:05:13.666 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster cluster1 osd new 53f419b1-d2db-459c-aa16-744363463a4f 2 2021-02-20T08:05:13.887 INFO:teuthology.orchestra.run.smithi003.stdout:2 2021-02-20T08:05:13.901 INFO:tasks.ceph.cluster1.osd.0:Restarting daemon 2021-02-20T08:05:14.022 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 0 2021-02-20T08:05:14.027 INFO:tasks.ceph.cluster1.osd.0:Started 2021-02-20T08:05:14.028 INFO:tasks.ceph.cluster1.osd.1:Restarting daemon 2021-02-20T08:05:14.028 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 1 2021-02-20T08:05:14.035 INFO:tasks.ceph.cluster1.osd.1:Started 2021-02-20T08:05:14.035 INFO:tasks.ceph.cluster1.osd.2:Restarting daemon 2021-02-20T08:05:14.036 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 2 2021-02-20T08:05:14.043 INFO:tasks.ceph.cluster1.osd.2:Started 2021-02-20T08:05:14.046 INFO:tasks.ceph:Waiting for OSDs to come up 2021-02-20T08:05:14.348 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster1 osd dump --format=json 2021-02-20T08:05:14.542 INFO:teuthology.misc.health.smithi003.stdout: 2021-02-20T08:05:14.542 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":5,"fsid":"5f712db1-9e20-4f92-8743-04e14bd4d26e","created":"2021-02-20T08:05:12.695613+0000","modified":"2021-02-20T08:05:13.885319+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"530535a2-f247-4671-9d81-66b354c8f8c1","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d07084ae-a43e-4df0-b4d4-616b0e830ba4","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"53f419b1-d2db-459c-aa16-744363463a4f","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T08:05:14.553 DEBUG:teuthology.misc:0 of 3 OSDs are up 2021-02-20T08:05:14.636 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T08:05:14.633+0000 7fb54a89f080 -1 Falling back to public interface 2021-02-20T08:05:14.648 INFO:tasks.ceph.cluster1.osd.0.smithi003.stderr:2021-02-20T08:05:14.645+0000 7fe1822a0080 -1 Falling back to public interface 2021-02-20T08:05:14.684 INFO:tasks.ceph.cluster1.osd.1.smithi003.stderr:2021-02-20T08:05:14.681+0000 7f4b65725080 -1 Falling back to public interface 2021-02-20T08:05:18.363 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T08:05:18.361+0000 7fb54a89f080 -1 osd.2 0 log_to_monitors {default=true} 2021-02-20T08:05:18.424 INFO:tasks.ceph.cluster1.osd.0.smithi003.stderr:2021-02-20T08:05:18.421+0000 7fe1822a0080 -1 osd.0 0 log_to_monitors {default=true} 2021-02-20T08:05:18.432 INFO:tasks.ceph.cluster1.osd.1.smithi003.stderr:2021-02-20T08:05:18.429+0000 7f4b65725080 -1 osd.1 0 log_to_monitors {default=true} 2021-02-20T08:05:19.716 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T08:05:19.713+0000 7fb541d68700 -1 osd.2 0 waiting for initial osdmap 2021-02-20T08:05:19.717 INFO:tasks.ceph.cluster1.osd.0.smithi003.stderr:2021-02-20T08:05:19.713+0000 7fe179769700 -1 osd.0 0 waiting for initial osdmap 2021-02-20T08:05:19.717 INFO:tasks.ceph.cluster1.osd.1.smithi003.stderr:2021-02-20T08:05:19.713+0000 7f4b5cbee700 -1 osd.1 0 waiting for initial osdmap 2021-02-20T08:05:20.857 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster1 osd dump --format=json 2021-02-20T08:05:21.163 INFO:teuthology.misc.health.smithi003.stdout: 2021-02-20T08:05:21.164 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":8,"fsid":"5f712db1-9e20-4f92-8743-04e14bd4d26e","created":"2021-02-20T08:05:12.695613+0000","modified":"2021-02-20T08:05:20.708077+0000","last_up_change":"2021-02-20T08:05:20.708077+0000","last_in_change":"2021-02-20T08:05:20.708077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"530535a2-f247-4671-9d81-66b354c8f8c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":15684},{"type":"v1","addr":"172.21.15.3:6809","nonce":15684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":15684},{"type":"v1","addr":"172.21.15.3:6811","nonce":15684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":15684},{"type":"v1","addr":"172.21.15.3:6815","nonce":15684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":15684},{"type":"v1","addr":"172.21.15.3:6813","nonce":15684}]},"public_addr":"172.21.15.3:6809/15684","cluster_addr":"172.21.15.3:6811/15684","heartbeat_back_addr":"172.21.15.3:6815/15684","heartbeat_front_addr":"172.21.15.3:6813/15684","state":["exists","up"]},{"osd":1,"uuid":"d07084ae-a43e-4df0-b4d4-616b0e830ba4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":15685},{"type":"v1","addr":"172.21.15.3:6817","nonce":15685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":15685},{"type":"v1","addr":"172.21.15.3:6819","nonce":15685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":15685},{"type":"v1","addr":"172.21.15.3:6823","nonce":15685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":15685},{"type":"v1","addr":"172.21.15.3:6821","nonce":15685}]},"public_addr":"172.21.15.3:6817/15685","cluster_addr":"172.21.15.3:6819/15685","heartbeat_back_addr":"172.21.15.3:6823/15685","heartbeat_front_addr":"172.21.15.3:6821/15685","state":["exists","up"]},{"osd":2,"uuid":"53f419b1-d2db-459c-aa16-744363463a4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":15686},{"type":"v1","addr":"172.21.15.3:6801","nonce":15686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":15686},{"type":"v1","addr":"172.21.15.3:6803","nonce":15686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":15686},{"type":"v1","addr":"172.21.15.3:6807","nonce":15686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":15686},{"type":"v1","addr":"172.21.15.3:6805","nonce":15686}]},"public_addr":"172.21.15.3:6801/15686","cluster_addr":"172.21.15.3:6803/15686","heartbeat_back_addr":"172.21.15.3:6807/15686","heartbeat_front_addr":"172.21.15.3:6805/15686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T08:05:21.176 DEBUG:teuthology.misc:3 of 3 OSDs are up 2021-02-20T08:05:21.177 INFO:tasks.ceph:Creating RBD pool 2021-02-20T08:05:21.177 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster cluster1 osd pool create rbd 8 2021-02-20T08:05:21.736 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'rbd' created 2021-02-20T08:05:21.750 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster cluster1 osd pool application enable rbd rbd --yes-i-really-mean-it 2021-02-20T08:05:22.760 INFO:teuthology.orchestra.run.smithi003.stderr:enabled application 'rbd' on pool 'rbd' 2021-02-20T08:05:22.774 INFO:tasks.ceph:Starting mds daemons in cluster cluster1... 2021-02-20T08:05:22.775 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2021-02-20T08:05:22.775 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2021-02-20T08:05:23.071 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:23.072 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":10,"fsid":"5f712db1-9e20-4f92-8743-04e14bd4d26e","created":"2021-02-20T08:05:12.695613+0000","modified":"2021-02-20T08:05:22.728981+0000","last_up_change":"2021-02-20T08:05:20.708077+0000","last_in_change":"2021-02-20T08:05:20.708077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-02-20T08:05:20.779644+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-02-20T08:05:21.548683+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"530535a2-f247-4671-9d81-66b354c8f8c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":15684},{"type":"v1","addr":"172.21.15.3:6809","nonce":15684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":15684},{"type":"v1","addr":"172.21.15.3:6811","nonce":15684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":15684},{"type":"v1","addr":"172.21.15.3:6815","nonce":15684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":15684},{"type":"v1","addr":"172.21.15.3:6813","nonce":15684}]},"public_addr":"172.21.15.3:6809/15684","cluster_addr":"172.21.15.3:6811/15684","heartbeat_back_addr":"172.21.15.3:6815/15684","heartbeat_front_addr":"172.21.15.3:6813/15684","state":["exists","up"]},{"osd":1,"uuid":"d07084ae-a43e-4df0-b4d4-616b0e830ba4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":15685},{"type":"v1","addr":"172.21.15.3:6817","nonce":15685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":15685},{"type":"v1","addr":"172.21.15.3:6819","nonce":15685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":15685},{"type":"v1","addr":"172.21.15.3:6823","nonce":15685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":15685},{"type":"v1","addr":"172.21.15.3:6821","nonce":15685}]},"public_addr":"172.21.15.3:6817/15685","cluster_addr":"172.21.15.3:6819/15685","heartbeat_back_addr":"172.21.15.3:6823/15685","heartbeat_front_addr":"172.21.15.3:6821/15685","state":["exists","up"]},{"osd":2,"uuid":"53f419b1-d2db-459c-aa16-744363463a4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":15686},{"type":"v1","addr":"172.21.15.3:6801","nonce":15686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":15686},{"type":"v1","addr":"172.21.15.3:6803","nonce":15686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":15686},{"type":"v1","addr":"172.21.15.3:6807","nonce":15686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":15686},{"type":"v1","addr":"172.21.15.3:6805","nonce":15686}]},"public_addr":"172.21.15.3:6801/15686","cluster_addr":"172.21.15.3:6803/15686","heartbeat_back_addr":"172.21.15.3:6807/15686","heartbeat_front_addr":"172.21.15.3:6805/15686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T08:05:19.444583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T08:05:19.438174+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T08:05:19.398418+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T08:05:23.086 INFO:tasks.ceph.ceph_manager.cluster1:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-02-20T08:05:20.779644+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-02-20T08:05:21.548683+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-20T08:05:23.087 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd pool get device_health_metrics pg_num 2021-02-20T08:05:23.395 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2021-02-20T08:05:23.411 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd pool get rbd pg_num 2021-02-20T08:05:23.714 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2021-02-20T08:05:23.729 INFO:tasks.ceph:Waiting until cluster1 daemons up and pgs clean... 2021-02-20T08:05:23.729 INFO:tasks.ceph.ceph_manager.cluster1:waiting for mgr available 2021-02-20T08:05:23.730 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 mgr dump --format=json 2021-02-20T08:05:24.072 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:24.086 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":4,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":15497},{"type":"v1","addr":"172.21.15.3:6825","nonce":15497}]},"active_addr":"172.21.15.3:6825/15497","active_change":"2021-02-20T08:05:17.633089+0000","active_mgr_features":4540138303579357183,"available":true,"standbys":[],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"haproxy","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":999165745}]},{"addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":672465233}]}]}} 2021-02-20T08:05:24.091 INFO:tasks.ceph.ceph_manager.cluster1:mgr available! 2021-02-20T08:05:24.091 INFO:tasks.ceph.ceph_manager.cluster1:waiting for all up 2021-02-20T08:05:24.091 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2021-02-20T08:05:24.389 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:24.390 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":11,"fsid":"5f712db1-9e20-4f92-8743-04e14bd4d26e","created":"2021-02-20T08:05:12.695613+0000","modified":"2021-02-20T08:05:23.738450+0000","last_up_change":"2021-02-20T08:05:20.708077+0000","last_in_change":"2021-02-20T08:05:20.708077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-02-20T08:05:20.779644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-02-20T08:05:21.548683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"530535a2-f247-4671-9d81-66b354c8f8c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":15684},{"type":"v1","addr":"172.21.15.3:6809","nonce":15684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":15684},{"type":"v1","addr":"172.21.15.3:6811","nonce":15684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":15684},{"type":"v1","addr":"172.21.15.3:6815","nonce":15684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":15684},{"type":"v1","addr":"172.21.15.3:6813","nonce":15684}]},"public_addr":"172.21.15.3:6809/15684","cluster_addr":"172.21.15.3:6811/15684","heartbeat_back_addr":"172.21.15.3:6815/15684","heartbeat_front_addr":"172.21.15.3:6813/15684","state":["exists","up"]},{"osd":1,"uuid":"d07084ae-a43e-4df0-b4d4-616b0e830ba4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":15685},{"type":"v1","addr":"172.21.15.3:6817","nonce":15685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":15685},{"type":"v1","addr":"172.21.15.3:6819","nonce":15685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":15685},{"type":"v1","addr":"172.21.15.3:6823","nonce":15685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":15685},{"type":"v1","addr":"172.21.15.3:6821","nonce":15685}]},"public_addr":"172.21.15.3:6817/15685","cluster_addr":"172.21.15.3:6819/15685","heartbeat_back_addr":"172.21.15.3:6823/15685","heartbeat_front_addr":"172.21.15.3:6821/15685","state":["exists","up"]},{"osd":2,"uuid":"53f419b1-d2db-459c-aa16-744363463a4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":15686},{"type":"v1","addr":"172.21.15.3:6801","nonce":15686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":15686},{"type":"v1","addr":"172.21.15.3:6803","nonce":15686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":15686},{"type":"v1","addr":"172.21.15.3:6807","nonce":15686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":15686},{"type":"v1","addr":"172.21.15.3:6805","nonce":15686}]},"public_addr":"172.21.15.3:6801/15686","cluster_addr":"172.21.15.3:6803/15686","heartbeat_back_addr":"172.21.15.3:6807/15686","heartbeat_front_addr":"172.21.15.3:6805/15686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T08:05:19.444583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T08:05:19.438174+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T08:05:19.398418+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T08:05:24.404 INFO:tasks.ceph.ceph_manager.cluster1:all up! 2021-02-20T08:05:24.405 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2021-02-20T08:05:24.706 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:24.706 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":11,"fsid":"5f712db1-9e20-4f92-8743-04e14bd4d26e","created":"2021-02-20T08:05:12.695613+0000","modified":"2021-02-20T08:05:23.738450+0000","last_up_change":"2021-02-20T08:05:20.708077+0000","last_in_change":"2021-02-20T08:05:20.708077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-02-20T08:05:20.779644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-02-20T08:05:21.548683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"530535a2-f247-4671-9d81-66b354c8f8c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":15684},{"type":"v1","addr":"172.21.15.3:6809","nonce":15684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":15684},{"type":"v1","addr":"172.21.15.3:6811","nonce":15684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":15684},{"type":"v1","addr":"172.21.15.3:6815","nonce":15684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":15684},{"type":"v1","addr":"172.21.15.3:6813","nonce":15684}]},"public_addr":"172.21.15.3:6809/15684","cluster_addr":"172.21.15.3:6811/15684","heartbeat_back_addr":"172.21.15.3:6815/15684","heartbeat_front_addr":"172.21.15.3:6813/15684","state":["exists","up"]},{"osd":1,"uuid":"d07084ae-a43e-4df0-b4d4-616b0e830ba4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":15685},{"type":"v1","addr":"172.21.15.3:6817","nonce":15685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":15685},{"type":"v1","addr":"172.21.15.3:6819","nonce":15685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":15685},{"type":"v1","addr":"172.21.15.3:6823","nonce":15685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":15685},{"type":"v1","addr":"172.21.15.3:6821","nonce":15685}]},"public_addr":"172.21.15.3:6817/15685","cluster_addr":"172.21.15.3:6819/15685","heartbeat_back_addr":"172.21.15.3:6823/15685","heartbeat_front_addr":"172.21.15.3:6821/15685","state":["exists","up"]},{"osd":2,"uuid":"53f419b1-d2db-459c-aa16-744363463a4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":15686},{"type":"v1","addr":"172.21.15.3:6801","nonce":15686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":15686},{"type":"v1","addr":"172.21.15.3:6803","nonce":15686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":15686},{"type":"v1","addr":"172.21.15.3:6807","nonce":15686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":15686},{"type":"v1","addr":"172.21.15.3:6805","nonce":15686}]},"public_addr":"172.21.15.3:6801/15686","cluster_addr":"172.21.15.3:6803/15686","heartbeat_back_addr":"172.21.15.3:6807/15686","heartbeat_front_addr":"172.21.15.3:6805/15686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T08:05:19.444583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T08:05:19.438174+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T08:05:19.398418+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T08:05:24.720 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.0 flush_pg_stats 2021-02-20T08:05:24.895 INFO:teuthology.orchestra.run.smithi003.stdout:34359738372 2021-02-20T08:05:24.896 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.1 flush_pg_stats 2021-02-20T08:05:25.077 INFO:teuthology.orchestra.run.smithi003.stdout:34359738370 2021-02-20T08:05:25.078 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.2 flush_pg_stats 2021-02-20T08:05:25.258 INFO:teuthology.orchestra.run.smithi003.stdout:34359738370 2021-02-20T08:05:25.258 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.0 2021-02-20T08:05:25.568 INFO:teuthology.orchestra.run.smithi003.stdout:34359738371 2021-02-20T08:05:25.583 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738372 got 34359738371 for osd.0 2021-02-20T08:05:26.584 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.0 2021-02-20T08:05:26.891 INFO:teuthology.orchestra.run.smithi003.stdout:34359738372 2021-02-20T08:05:26.907 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738372 got 34359738372 for osd.0 2021-02-20T08:05:26.908 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.1 2021-02-20T08:05:27.214 INFO:teuthology.orchestra.run.smithi003.stdout:34359738370 2021-02-20T08:05:27.228 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738370 got 34359738370 for osd.1 2021-02-20T08:05:27.229 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.2 2021-02-20T08:05:27.536 INFO:teuthology.orchestra.run.smithi003.stdout:34359738370 2021-02-20T08:05:27.551 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738370 got 34359738370 for osd.2 2021-02-20T08:05:27.552 INFO:tasks.ceph.ceph_manager.cluster1:waiting for clean 2021-02-20T08:05:27.552 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2021-02-20T08:05:27.850 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:27.853 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2021-02-20T08:05:27.865 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2021-02-20T08:05:27.640305+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":14648,"kb_used_data":288,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":283100872,"statfs":{"total":289910292480,"available":289895292928,"internally_reserved":0,"allocated":294912,"data_stored":80052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:22.760732+0000","last_change":"2021-02-20T08:05:22.759090+0000","last_active":"2021-02-20T08:05:22.760732+0000","last_peered":"2021-02-20T08:05:22.760732+0000","last_clean":"2021-02-20T08:05:22.760732+0000","last_became_active":"2021-02-20T08:05:22.758713+0000","last_became_peered":"2021-02-20T08:05:22.758713+0000","last_unstale":"2021-02-20T08:05:22.760732+0000","last_undegraded":"2021-02-20T08:05:22.760732+0000","last_fullsized":"2021-02-20T08:05:22.760732+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_clean_scrub_stamp":"2021-02-20T08:05:21.722547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:22.760684+0000","last_change":"2021-02-20T08:05:22.758233+0000","last_active":"2021-02-20T08:05:22.760684+0000","last_peered":"2021-02-20T08:05:22.760684+0000","last_clean":"2021-02-20T08:05:22.760684+0000","last_became_active":"2021-02-20T08:05:22.757676+0000","last_became_peered":"2021-02-20T08:05:22.757676+0000","last_unstale":"2021-02-20T08:05:22.760684+0000","last_undegraded":"2021-02-20T08:05:22.760684+0000","last_fullsized":"2021-02-20T08:05:22.760684+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_clean_scrub_stamp":"2021-02-20T08:05:21.722547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:22.761400+0000","last_change":"2021-02-20T08:05:22.760397+0000","last_active":"2021-02-20T08:05:22.761400+0000","last_peered":"2021-02-20T08:05:22.761400+0000","last_clean":"2021-02-20T08:05:22.761400+0000","last_became_active":"2021-02-20T08:05:22.759915+0000","last_became_peered":"2021-02-20T08:05:22.759915+0000","last_unstale":"2021-02-20T08:05:22.761400+0000","last_undegraded":"2021-02-20T08:05:22.761400+0000","last_fullsized":"2021-02-20T08:05:22.761400+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_clean_scrub_stamp":"2021-02-20T08:05:21.722547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:22.761349+0000","last_change":"2021-02-20T08:05:22.760543+0000","last_active":"2021-02-20T08:05:22.761349+0000","last_peered":"2021-02-20T08:05:22.761349+0000","last_clean":"2021-02-20T08:05:22.761349+0000","last_became_active":"2021-02-20T08:05:22.760145+0000","last_became_peered":"2021-02-20T08:05:22.760145+0000","last_unstale":"2021-02-20T08:05:22.761349+0000","last_undegraded":"2021-02-20T08:05:22.761349+0000","last_fullsized":"2021-02-20T08:05:22.761349+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_clean_scrub_stamp":"2021-02-20T08:05:21.722547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:22.760693+0000","last_change":"2021-02-20T08:05:22.758750+0000","last_active":"2021-02-20T08:05:22.760693+0000","last_peered":"2021-02-20T08:05:22.760693+0000","last_clean":"2021-02-20T08:05:22.760693+0000","last_became_active":"2021-02-20T08:05:22.758358+0000","last_became_peered":"2021-02-20T08:05:22.758358+0000","last_unstale":"2021-02-20T08:05:22.760693+0000","last_undegraded":"2021-02-20T08:05:22.760693+0000","last_fullsized":"2021-02-20T08:05:22.760693+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_clean_scrub_stamp":"2021-02-20T08:05:21.722547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:22.762312+0000","last_change":"2021-02-20T08:05:22.760566+0000","last_active":"2021-02-20T08:05:22.762312+0000","last_peered":"2021-02-20T08:05:22.762312+0000","last_clean":"2021-02-20T08:05:22.762312+0000","last_became_active":"2021-02-20T08:05:22.760056+0000","last_became_peered":"2021-02-20T08:05:22.760056+0000","last_unstale":"2021-02-20T08:05:22.762312+0000","last_undegraded":"2021-02-20T08:05:22.762312+0000","last_fullsized":"2021-02-20T08:05:22.762312+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_clean_scrub_stamp":"2021-02-20T08:05:21.722547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:22.759559+0000","last_change":"2021-02-20T08:05:22.758648+0000","last_active":"2021-02-20T08:05:22.759559+0000","last_peered":"2021-02-20T08:05:22.759559+0000","last_clean":"2021-02-20T08:05:22.759559+0000","last_became_active":"2021-02-20T08:05:22.758267+0000","last_became_peered":"2021-02-20T08:05:22.758267+0000","last_unstale":"2021-02-20T08:05:22.759559+0000","last_undegraded":"2021-02-20T08:05:22.759559+0000","last_fullsized":"2021-02-20T08:05:22.759559+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_clean_scrub_stamp":"2021-02-20T08:05:21.722547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:22.765202+0000","last_change":"2021-02-20T08:05:22.764031+0000","last_active":"2021-02-20T08:05:22.765202+0000","last_peered":"2021-02-20T08:05:22.765202+0000","last_clean":"2021-02-20T08:05:22.765202+0000","last_became_active":"2021-02-20T08:05:22.763211+0000","last_became_peered":"2021-02-20T08:05:22.763211+0000","last_unstale":"2021-02-20T08:05:22.765202+0000","last_undegraded":"2021-02-20T08:05:22.765202+0000","last_fullsized":"2021-02-20T08:05:22.765202+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_clean_scrub_stamp":"2021-02-20T08:05:21.722547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:22.759507+0000","last_change":"2021-02-20T08:05:22.757224+0000","last_active":"2021-02-20T08:05:22.759507+0000","last_peered":"2021-02-20T08:05:22.759507+0000","last_clean":"2021-02-20T08:05:22.759507+0000","last_became_active":"2021-02-20T08:05:22.756400+0000","last_became_peered":"2021-02-20T08:05:22.756400+0000","last_unstale":"2021-02-20T08:05:22.759507+0000","last_undegraded":"2021-02-20T08:05:22.759507+0000","last_fullsized":"2021-02-20T08:05:22.759507+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_clean_scrub_stamp":"2021-02-20T08:05:21.722547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4840,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94367000,"statfs":{"total":96636764160,"available":96631808000,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4630000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.657}]}]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4930000000000001}]}]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]}]}]}} 2021-02-20T08:05:27.867 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2021-02-20T08:05:28.171 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:28.174 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2021-02-20T08:05:28.185 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2021-02-20T08:05:27.640305+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":14648,"kb_used_data":288,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":283100872,"statfs":{"total":289910292480,"available":289895292928,"internally_reserved":0,"allocated":294912,"data_stored":80052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:22.760732+0000","last_change":"2021-02-20T08:05:22.759090+0000","last_active":"2021-02-20T08:05:22.760732+0000","last_peered":"2021-02-20T08:05:22.760732+0000","last_clean":"2021-02-20T08:05:22.760732+0000","last_became_active":"2021-02-20T08:05:22.758713+0000","last_became_peered":"2021-02-20T08:05:22.758713+0000","last_unstale":"2021-02-20T08:05:22.760732+0000","last_undegraded":"2021-02-20T08:05:22.760732+0000","last_fullsized":"2021-02-20T08:05:22.760732+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_clean_scrub_stamp":"2021-02-20T08:05:21.722547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:22.760684+0000","last_change":"2021-02-20T08:05:22.758233+0000","last_active":"2021-02-20T08:05:22.760684+0000","last_peered":"2021-02-20T08:05:22.760684+0000","last_clean":"2021-02-20T08:05:22.760684+0000","last_became_active":"2021-02-20T08:05:22.757676+0000","last_became_peered":"2021-02-20T08:05:22.757676+0000","last_unstale":"2021-02-20T08:05:22.760684+0000","last_undegraded":"2021-02-20T08:05:22.760684+0000","last_fullsized":"2021-02-20T08:05:22.760684+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_clean_scrub_stamp":"2021-02-20T08:05:21.722547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:22.761400+0000","last_change":"2021-02-20T08:05:22.760397+0000","last_active":"2021-02-20T08:05:22.761400+0000","last_peered":"2021-02-20T08:05:22.761400+0000","last_clean":"2021-02-20T08:05:22.761400+0000","last_became_active":"2021-02-20T08:05:22.759915+0000","last_became_peered":"2021-02-20T08:05:22.759915+0000","last_unstale":"2021-02-20T08:05:22.761400+0000","last_undegraded":"2021-02-20T08:05:22.761400+0000","last_fullsized":"2021-02-20T08:05:22.761400+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_clean_scrub_stamp":"2021-02-20T08:05:21.722547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:22.761349+0000","last_change":"2021-02-20T08:05:22.760543+0000","last_active":"2021-02-20T08:05:22.761349+0000","last_peered":"2021-02-20T08:05:22.761349+0000","last_clean":"2021-02-20T08:05:22.761349+0000","last_became_active":"2021-02-20T08:05:22.760145+0000","last_became_peered":"2021-02-20T08:05:22.760145+0000","last_unstale":"2021-02-20T08:05:22.761349+0000","last_undegraded":"2021-02-20T08:05:22.761349+0000","last_fullsized":"2021-02-20T08:05:22.761349+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_clean_scrub_stamp":"2021-02-20T08:05:21.722547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:22.760693+0000","last_change":"2021-02-20T08:05:22.758750+0000","last_active":"2021-02-20T08:05:22.760693+0000","last_peered":"2021-02-20T08:05:22.760693+0000","last_clean":"2021-02-20T08:05:22.760693+0000","last_became_active":"2021-02-20T08:05:22.758358+0000","last_became_peered":"2021-02-20T08:05:22.758358+0000","last_unstale":"2021-02-20T08:05:22.760693+0000","last_undegraded":"2021-02-20T08:05:22.760693+0000","last_fullsized":"2021-02-20T08:05:22.760693+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_clean_scrub_stamp":"2021-02-20T08:05:21.722547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:22.762312+0000","last_change":"2021-02-20T08:05:22.760566+0000","last_active":"2021-02-20T08:05:22.762312+0000","last_peered":"2021-02-20T08:05:22.762312+0000","last_clean":"2021-02-20T08:05:22.762312+0000","last_became_active":"2021-02-20T08:05:22.760056+0000","last_became_peered":"2021-02-20T08:05:22.760056+0000","last_unstale":"2021-02-20T08:05:22.762312+0000","last_undegraded":"2021-02-20T08:05:22.762312+0000","last_fullsized":"2021-02-20T08:05:22.762312+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_clean_scrub_stamp":"2021-02-20T08:05:21.722547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:22.759559+0000","last_change":"2021-02-20T08:05:22.758648+0000","last_active":"2021-02-20T08:05:22.759559+0000","last_peered":"2021-02-20T08:05:22.759559+0000","last_clean":"2021-02-20T08:05:22.759559+0000","last_became_active":"2021-02-20T08:05:22.758267+0000","last_became_peered":"2021-02-20T08:05:22.758267+0000","last_unstale":"2021-02-20T08:05:22.759559+0000","last_undegraded":"2021-02-20T08:05:22.759559+0000","last_fullsized":"2021-02-20T08:05:22.759559+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_clean_scrub_stamp":"2021-02-20T08:05:21.722547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:22.765202+0000","last_change":"2021-02-20T08:05:22.764031+0000","last_active":"2021-02-20T08:05:22.765202+0000","last_peered":"2021-02-20T08:05:22.765202+0000","last_clean":"2021-02-20T08:05:22.765202+0000","last_became_active":"2021-02-20T08:05:22.763211+0000","last_became_peered":"2021-02-20T08:05:22.763211+0000","last_unstale":"2021-02-20T08:05:22.765202+0000","last_undegraded":"2021-02-20T08:05:22.765202+0000","last_fullsized":"2021-02-20T08:05:22.765202+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_clean_scrub_stamp":"2021-02-20T08:05:21.722547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:22.759507+0000","last_change":"2021-02-20T08:05:22.757224+0000","last_active":"2021-02-20T08:05:22.759507+0000","last_peered":"2021-02-20T08:05:22.759507+0000","last_clean":"2021-02-20T08:05:22.759507+0000","last_became_active":"2021-02-20T08:05:22.756400+0000","last_became_peered":"2021-02-20T08:05:22.756400+0000","last_unstale":"2021-02-20T08:05:22.759507+0000","last_undegraded":"2021-02-20T08:05:22.759507+0000","last_fullsized":"2021-02-20T08:05:22.759507+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:21.722547+0000","last_clean_scrub_stamp":"2021-02-20T08:05:21.722547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4840,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94367000,"statfs":{"total":96636764160,"available":96631808000,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4630000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.657}]}]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4930000000000001}]}]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]}]}]}} 2021-02-20T08:05:28.186 INFO:tasks.ceph.ceph_manager.cluster1:clean! 2021-02-20T08:05:28.187 INFO:tasks.ceph:Waiting until ceph cluster cluster1 is healthy... 2021-02-20T08:05:28.187 INFO:tasks.ceph.ceph_manager.cluster1:wait_until_healthy 2021-02-20T08:05:28.187 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 health --format=json 2021-02-20T08:05:28.540 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:28.541 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2021-02-20T08:05:28.555 INFO:tasks.ceph.ceph_manager.cluster1:wait_until_healthy done 2021-02-20T08:05:28.556 INFO:teuthology.run_tasks:Running task ceph... 2021-02-20T08:05:28.565 INFO:tasks.ceph:Creating ceph cluster cluster2... 2021-02-20T08:05:28.566 INFO:tasks.ceph:config {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'none'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(OSD_SLOW_PING_TIME'], 'cpu_profile': set(), 'cluster': 'cluster2', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2021-02-20T08:05:28.566 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi/5897918', 'branch': 'wip-mgolub-testing', 'description': 'rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}', 'email': 'to.my.trociny@gmail.com', 'first_in_suite': False, 'job_id': '5897918', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': 'run two ceph clusters and install rbd-mirror'}, {'desc': 'configure the permissions for client.mirror'}, {'desc': '2 ceph clusters with 1 mon and 3 osds each'}, {'desc': 'run four rbd-mirror daemons per cluster'}, {'desc': 'run the rbd_mirror_stress.sh workunit to test the rbd-mirror daemon'}], 'name': 'trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '18.04', 'overrides': {'admin_socket': {'branch': 'wip-mgolub-testing'}, 'ceph': {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'none'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(OSD_SLOW_PING_TIME'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mgolub-testing', 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}}, 'owner': 'scheduled_trociny@teuthology', 'priority': 50, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'], ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']], 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b', 'sleep_before_teardown': 0, 'suite': 'rbd', 'suite_branch': 'wip-mgolub-testing', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_d318d42d502ea13c78f2af2499efd19dcc7e174b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b', 'targets': {'smithi003.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyx++l8vCCc5LmHtTy8pQxD4u432ooKGE/AHMpohypjtB5vWw9eqB1m8w0iOU+wGGonfdzP49qcnKxw7NUtQh2vSkRA2kXw+sz6IzEsm7eqVV2UypIjsbl09hZB4m94uYSl5urqZ3IbVMVYUDtH+NAU8I+ghkhkTRSnb8Lc4bQmOiKDRyNXAa0mdofbiTcj32PSs8iWuDN2imSWHlUeBFjOvDKD1IGBic/qXPgTjWhDVTGyyT1GNM1Sv5pTbmpxLyxjm1qTMHVpU1nFgQTlUW4AJdmqF5ASBFlNJJjCxGguxlXAqqCbbz5cgzYnB1bwf7IzOtHBonnYp7ydVH0JDjB', 'smithi090.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJHRcfKBV/OkZP8o1/N7APaJg3UhsSRUYymD9kh3n2b7ZdoWr7TS7S8ZaUjR/zdYkdiX/JxbVf/WciS0KJ02FYSNpPTMHOgeqyw1Ih37MiOXKzwsg1PxdZmiqyNGkAlqr5pY61dA6xWSPgmvbvrmZlHkX97KC5EBl7gsvJokmA3YidM5VbGerWOLI/bRmbpzi8wJh9Ovm+rbln7Es0Jh7QBARXsFU9QNocjPWXhbHv7049UrfKUbAfiyBRpyFx6cePrOWl5P7lqO1h90JO8LxUa5c8ByjRGGE7PcMFIcQq0TyGAgSPnLdU4b6UM3z6FNPKL4pxeR1CuyiHjS5JF7XL'}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': ['rbd-mirror'], 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}}, {'ceph': {'cluster': 'cluster1', 'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'none'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(OSD_SLOW_PING_TIME'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}}, {'ceph': {'cluster': 'cluster2', 'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'none'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(OSD_SLOW_PING_TIME'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}}, {'exec': {'cluster1.client.mirror.0': ["sudo ceph --cluster cluster1 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'"]}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.3', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.3', 'thrash': True}}, {'rbd-mirror-thrash': {'cluster': 'cluster1'}}, {'rbd-mirror-thrash': {'cluster': 'cluster2'}}, {'workunit': {'clients': {'cluster1.client.mirror': ['rbd/rbd_mirror_stress.sh']}, 'env': {'CEPH_ARGS': '', 'MIRROR_IMAGE_MODE': 'snapshot', 'MIRROR_POOL_MODE': 'image', 'RBD_IMAGE_FEATURES': 'layering,exclusive-lock,object-map,fast-diff', 'RBD_MIRROR_INSTANCES': '4', 'RBD_MIRROR_USE_EXISTING_CLUSTER': '1', 'RBD_MIRROR_USE_RBD_MIRROR': '1'}, 'timeout': '6h'}}], 'teuthology_branch': 'master', 'tube': 'smithi', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.29353'} 2021-02-20T08:05:28.567 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/cluster2.data 2021-02-20T08:05:28.570 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/cluster2.data 2021-02-20T08:05:28.577 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/run/ceph 2021-02-20T08:05:28.620 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m0777 -- /var/run/ceph 2021-02-20T08:05:28.635 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:28.636 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/scratch_devs of=/dev/stdout 2021-02-20T08:05:28.688 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-02-20T08:05:28.688 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-02-20T08:05:28.737 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-02-20T08:05:28.737 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T08:05:28.737 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 341 Links: 1 2021-02-20T08:05:28.737 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T08:05:28.738 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2021-02-20 08:02:21.261158835 +0000 2021-02-20T08:05:28.738 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2021-02-20 08:02:21.257160835 +0000 2021-02-20T08:05:28.738 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2021-02-20 08:02:21.257160835 +0000 2021-02-20T08:05:28.738 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: - 2021-02-20T08:05:28.747 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2021-02-20T08:05:28.747 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2021-02-20T08:05:28.747 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000365166 s, 1.4 MB/s 2021-02-20T08:05:28.751 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-02-20T08:05:28.800 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-02-20T08:05:28.801 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T08:05:28.801 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 347 Links: 1 2021-02-20T08:05:28.801 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T08:05:28.802 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2021-02-20 08:02:21.592992827 +0000 2021-02-20T08:05:28.802 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2021-02-20 08:02:21.477050830 +0000 2021-02-20T08:05:28.802 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2021-02-20 08:02:21.477050830 +0000 2021-02-20T08:05:28.802 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: - 2021-02-20T08:05:28.810 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2021-02-20T08:05:28.810 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2021-02-20T08:05:28.811 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000330867 s, 1.5 MB/s 2021-02-20T08:05:28.814 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-02-20T08:05:28.864 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-02-20T08:05:28.865 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T08:05:28.865 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 353 Links: 1 2021-02-20T08:05:28.865 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T08:05:28.866 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2021-02-20 08:02:21.712932825 +0000 2021-02-20T08:05:28.866 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2021-02-20 08:02:21.708934824 +0000 2021-02-20T08:05:28.866 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2021-02-20 08:02:21.708934824 +0000 2021-02-20T08:05:28.866 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: - 2021-02-20T08:05:28.874 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2021-02-20T08:05:28.874 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2021-02-20T08:05:28.875 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000398484 s, 1.3 MB/s 2021-02-20T08:05:28.879 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-02-20T08:05:28.929 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-02-20T08:05:28.930 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T08:05:28.930 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 359 Links: 1 2021-02-20T08:05:28.930 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T08:05:28.931 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2021-02-20 08:02:22.044766816 +0000 2021-02-20T08:05:28.931 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2021-02-20 08:02:21.948814819 +0000 2021-02-20T08:05:28.931 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2021-02-20 08:02:21.948814819 +0000 2021-02-20T08:05:28.931 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: - 2021-02-20T08:05:28.939 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2021-02-20T08:05:28.940 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2021-02-20T08:05:28.940 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000414503 s, 1.2 MB/s 2021-02-20T08:05:28.944 INFO:tasks.ceph:osd dev map: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T08:05:28.944 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi090.front.sepia.ceph.com'): {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'}} 2021-02-20T08:05:28.944 INFO:tasks.ceph:Generating config... 2021-02-20T08:05:28.950 INFO:tasks.ceph:[client] debug rbd = 20 2021-02-20T08:05:28.950 INFO:tasks.ceph:[client] debug rbd_mirror = 15 2021-02-20T08:05:28.951 INFO:tasks.ceph:[client] log to stderr = False 2021-02-20T08:05:28.951 INFO:tasks.ceph:[client] rbd default features = 125 2021-02-20T08:05:28.952 INFO:tasks.ceph:[client] rbd mirror image policy type = none 2021-02-20T08:05:28.952 INFO:tasks.ceph:[client.mirror.0] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:05:28.953 INFO:tasks.ceph:[client.mirror.0] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:05:28.953 INFO:tasks.ceph:[client.mirror.1] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:05:28.953 INFO:tasks.ceph:[client.mirror.1] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:05:28.954 INFO:tasks.ceph:[client.mirror.2] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:05:28.954 INFO:tasks.ceph:[client.mirror.2] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:05:28.954 INFO:tasks.ceph:[client.mirror.3] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:05:28.954 INFO:tasks.ceph:[client.mirror.3] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:05:28.955 INFO:tasks.ceph:[global] mon client directed command retry = 5 2021-02-20T08:05:28.955 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2021-02-20T08:05:28.955 INFO:tasks.ceph:[mgr] debug mgr = 20 2021-02-20T08:05:28.956 INFO:tasks.ceph:[mgr] debug ms = 1 2021-02-20T08:05:28.956 INFO:tasks.ceph:[mon] debug mon = 20 2021-02-20T08:05:28.956 INFO:tasks.ceph:[mon] debug ms = 1 2021-02-20T08:05:28.957 INFO:tasks.ceph:[mon] debug paxos = 20 2021-02-20T08:05:28.957 INFO:tasks.ceph:[osd] bdev async discard = True 2021-02-20T08:05:28.957 INFO:tasks.ceph:[osd] bdev enable discard = True 2021-02-20T08:05:28.957 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2021-02-20T08:05:28.958 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2021-02-20T08:05:28.958 INFO:tasks.ceph:[osd] debug bluefs = 20 2021-02-20T08:05:28.958 INFO:tasks.ceph:[osd] debug bluestore = 20 2021-02-20T08:05:28.959 INFO:tasks.ceph:[osd] debug filestore = 20 2021-02-20T08:05:28.959 INFO:tasks.ceph:[osd] debug journal = 20 2021-02-20T08:05:28.959 INFO:tasks.ceph:[osd] debug ms = 20 2021-02-20T08:05:28.960 INFO:tasks.ceph:[osd] debug osd = 25 2021-02-20T08:05:28.960 INFO:tasks.ceph:[osd] debug rocksdb = 10 2021-02-20T08:05:28.960 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2021-02-20T08:05:28.960 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2021-02-20T08:05:28.961 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2021-02-20T08:05:28.961 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2021-02-20T08:05:28.961 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2021-02-20T08:05:28.962 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2021-02-20T08:05:28.962 INFO:tasks.ceph:Setting up cluster2.mon.a... 2021-02-20T08:05:28.962 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/cluster2.keyring 2021-02-20T08:05:29.014 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/cluster2.keyring 2021-02-20T08:05:29.017 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/cluster2.keyring 2021-02-20T08:05:29.092 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2021-02-20T08:05:29.149 DEBUG:tasks.ceph:Ceph mon addresses: [('cluster2.mon.a', '172.21.15.90')] 2021-02-20T08:05:29.149 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.90', 'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'none'}, 'mon.a': {}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}} 2021-02-20T08:05:29.151 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:29.151 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2021-02-20T08:05:29.196 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.90 --print /home/ubuntu/cephtest/cluster2.monmap 2021-02-20T08:05:29.269 INFO:teuthology.orchestra.run.smithi090.stdout:monmaptool: monmap file /home/ubuntu/cephtest/cluster2.monmap 2021-02-20T08:05:29.269 INFO:teuthology.orchestra.run.smithi090.stdout:monmaptool: generated fsid 58fc22eb-10c4-4442-9a61-4d595829e8b5 2021-02-20T08:05:29.269 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 0 2021-02-20T08:05:29.270 INFO:teuthology.orchestra.run.smithi090.stdout:fsid 58fc22eb-10c4-4442-9a61-4d595829e8b5 2021-02-20T08:05:29.270 INFO:teuthology.orchestra.run.smithi090.stdout:last_changed 2021-02-20T08:05:29.270659+0000 2021-02-20T08:05:29.270 INFO:teuthology.orchestra.run.smithi090.stdout:created 2021-02-20T08:05:29.270659+0000 2021-02-20T08:05:29.271 INFO:teuthology.orchestra.run.smithi090.stdout:min_mon_release 0 (unknown) 2021-02-20T08:05:29.271 INFO:teuthology.orchestra.run.smithi090.stdout:election_strategy: 1 2021-02-20T08:05:29.271 INFO:teuthology.orchestra.run.smithi090.stdout:0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2021-02-20T08:05:29.272 INFO:teuthology.orchestra.run.smithi090.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/cluster2.monmap (1 monitors) 2021-02-20T08:05:29.273 DEBUG:teuthology.orchestra.run.smithi090:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2021-02-20T08:05:29.320 INFO:tasks.ceph:Writing /etc/ceph/cluster2.conf for FSID 58fc22eb-10c4-4442-9a61-4d595829e8b5... 2021-02-20T08:05:29.321 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster2.conf && sudo chmod 0644 /etc/ceph/cluster2.conf > /dev/null 2021-02-20T08:05:29.324 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster2.conf && sudo chmod 0644 /etc/ceph/cluster2.conf > /dev/null 2021-02-20T08:05:29.365 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2021-02-20T08:05:29.366 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2021-02-20T08:05:29.366 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-20T08:05:29.367 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2021-02-20T08:05:29.367 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.367 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2021-02-20T08:05:29.367 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.367 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2021-02-20T08:05:29.368 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.368 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2021-02-20T08:05:29.368 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2021-02-20T08:05:29.368 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.368 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2021-02-20T08:05:29.369 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2021-02-20T08:05:29.369 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.369 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-20T08:05:29.370 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2021-02-20T08:05:29.371 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.371 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2021-02-20T08:05:29.371 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2021-02-20T08:05:29.371 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.372 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2021-02-20T08:05:29.372 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.372 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2021-02-20T08:05:29.372 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2021-02-20T08:05:29.372 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2021-02-20T08:05:29.373 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2021-02-20T08:05:29.373 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2021-02-20T08:05:29.373 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2021-02-20T08:05:29.373 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2021-02-20T08:05:29.374 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-20T08:05:29.374 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-20T08:05:29.374 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2021-02-20T08:05:29.374 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.374 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-20T08:05:29.375 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.375 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2021-02-20T08:05:29.375 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.375 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2021-02-20T08:05:29.376 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.376 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2021-02-20T08:05:29.377 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2021-02-20T08:05:29.378 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2021-02-20T08:05:29.378 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = 172.21.15.90 2021-02-20T08:05:29.378 INFO:teuthology.orchestra.run.smithi003.stdout: mon client directed command retry = 5 2021-02-20T08:05:29.378 INFO:teuthology.orchestra.run.smithi003.stdout: ms inject socket failures = 5000 2021-02-20T08:05:29.379 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 58fc22eb-10c4-4442-9a61-4d595829e8b5 2021-02-20T08:05:29.379 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.379 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2021-02-20T08:05:29.379 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2021-02-20T08:05:29.379 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.380 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2021-02-20T08:05:29.380 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2021-02-20T08:05:29.380 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.380 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2021-02-20T08:05:29.380 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2021-02-20T08:05:29.381 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.381 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2021-02-20T08:05:29.381 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2021-02-20T08:05:29.381 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2021-02-20T08:05:29.381 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.382 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2021-02-20T08:05:29.382 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2021-02-20T08:05:29.382 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.382 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2021-02-20T08:05:29.383 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.383 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2021-02-20T08:05:29.383 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.383 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2021-02-20T08:05:29.383 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.384 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2021-02-20T08:05:29.384 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2021-02-20T08:05:29.384 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.384 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2021-02-20T08:05:29.384 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2021-02-20T08:05:29.385 INFO:teuthology.orchestra.run.smithi003.stdout: bdev async discard = True 2021-02-20T08:05:29.385 INFO:teuthology.orchestra.run.smithi003.stdout: bdev enable discard = True 2021-02-20T08:05:29.385 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2021-02-20T08:05:29.385 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2021-02-20T08:05:29.385 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 20 2021-02-20T08:05:29.386 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 20 2021-02-20T08:05:29.386 INFO:teuthology.orchestra.run.smithi003.stdout: debug filestore = 20 2021-02-20T08:05:29.387 INFO:teuthology.orchestra.run.smithi003.stdout: debug journal = 20 2021-02-20T08:05:29.387 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 20 2021-02-20T08:05:29.387 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 25 2021-02-20T08:05:29.387 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 10 2021-02-20T08:05:29.388 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2021-02-20T08:05:29.388 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2021-02-20T08:05:29.388 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2021-02-20T08:05:29.388 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2021-02-20T08:05:29.389 INFO:teuthology.orchestra.run.smithi003.stdout: osd memory target = 1610612736 2021-02-20T08:05:29.389 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2021-02-20T08:05:29.389 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.389 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2021-02-20T08:05:29.390 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2021-02-20T08:05:29.390 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2021-02-20T08:05:29.390 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2021-02-20T08:05:29.391 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2021-02-20T08:05:29.391 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2021-02-20T08:05:29.391 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2021-02-20T08:05:29.391 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2021-02-20T08:05:29.391 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.392 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2021-02-20T08:05:29.392 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2021-02-20T08:05:29.392 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2021-02-20T08:05:29.392 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2021-02-20T08:05:29.393 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2021-02-20T08:05:29.393 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2021-02-20T08:05:29.393 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2021-02-20T08:05:29.393 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2021-02-20T08:05:29.394 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2021-02-20T08:05:29.394 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2021-02-20T08:05:29.394 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2021-02-20T08:05:29.394 INFO:teuthology.orchestra.run.smithi003.stdout: 2021-02-20T08:05:29.394 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2021-02-20T08:05:29.395 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2021-02-20T08:05:29.395 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2021-02-20T08:05:29.395 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2021-02-20T08:05:29.395 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-20T08:05:29.395 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-20T08:05:29.396 INFO:teuthology.orchestra.run.smithi003.stdout: debug rbd = 20 2021-02-20T08:05:29.396 INFO:teuthology.orchestra.run.smithi003.stdout: debug rbd_mirror = 15 2021-02-20T08:05:29.396 INFO:teuthology.orchestra.run.smithi003.stdout: log to stderr = False 2021-02-20T08:05:29.396 INFO:teuthology.orchestra.run.smithi003.stdout: rbd default features = 125 2021-02-20T08:05:29.397 INFO:teuthology.orchestra.run.smithi003.stdout: rbd mirror image policy type = none 2021-02-20T08:05:29.397 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2021-02-20T08:05:29.397 INFO:teuthology.orchestra.run.smithi003.stdout:[client.mirror.0] 2021-02-20T08:05:29.397 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:05:29.397 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:05:29.398 INFO:teuthology.orchestra.run.smithi003.stdout:[client.mirror.1] 2021-02-20T08:05:29.398 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:05:29.398 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:05:29.398 INFO:teuthology.orchestra.run.smithi003.stdout:[client.mirror.2] 2021-02-20T08:05:29.398 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:05:29.399 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:05:29.399 INFO:teuthology.orchestra.run.smithi003.stdout:[client.mirror.3] 2021-02-20T08:05:29.399 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:05:29.399 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:05:29.400 INFO:teuthology.orchestra.run.smithi090.stdout:[global] 2021-02-20T08:05:29.401 INFO:teuthology.orchestra.run.smithi090.stdout: chdir = "" 2021-02-20T08:05:29.401 INFO:teuthology.orchestra.run.smithi090.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-20T08:05:29.401 INFO:teuthology.orchestra.run.smithi090.stdout: auth supported = cephx 2021-02-20T08:05:29.401 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.402 INFO:teuthology.orchestra.run.smithi090.stdout: filestore xattr use omap = true 2021-02-20T08:05:29.402 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.402 INFO:teuthology.orchestra.run.smithi090.stdout: mon clock drift allowed = 1.000 2021-02-20T08:05:29.402 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.402 INFO:teuthology.orchestra.run.smithi090.stdout: osd crush chooseleaf type = 0 2021-02-20T08:05:29.403 INFO:teuthology.orchestra.run.smithi090.stdout: auth debug = true 2021-02-20T08:05:29.403 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.403 INFO:teuthology.orchestra.run.smithi090.stdout: ms die on old message = true 2021-02-20T08:05:29.403 INFO:teuthology.orchestra.run.smithi090.stdout: ms die on bug = true 2021-02-20T08:05:29.404 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.404 INFO:teuthology.orchestra.run.smithi090.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-20T08:05:29.404 INFO:teuthology.orchestra.run.smithi090.stdout: mon pg warn max object skew = 0 2021-02-20T08:05:29.404 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.404 INFO:teuthology.orchestra.run.smithi090.stdout: # disable pg_autoscaler by default for new pools 2021-02-20T08:05:29.405 INFO:teuthology.orchestra.run.smithi090.stdout: osd_pool_default_pg_autoscale_mode = off 2021-02-20T08:05:29.405 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.405 INFO:teuthology.orchestra.run.smithi090.stdout: osd pool default size = 2 2021-02-20T08:05:29.406 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.406 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd allow primary affinity = true 2021-02-20T08:05:29.406 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd allow pg remap = true 2021-02-20T08:05:29.406 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on legacy crush tunables = false 2021-02-20T08:05:29.406 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on crush straw calc version zero = false 2021-02-20T08:05:29.407 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on no sortbitwise = false 2021-02-20T08:05:29.407 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on osd down out interval zero = false 2021-02-20T08:05:29.407 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on too few osds = false 2021-02-20T08:05:29.408 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-20T08:05:29.408 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-20T08:05:29.408 INFO:teuthology.orchestra.run.smithi090.stdout: mon_allow_pool_size_one = true 2021-02-20T08:05:29.408 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.408 INFO:teuthology.orchestra.run.smithi090.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-20T08:05:29.409 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.409 INFO:teuthology.orchestra.run.smithi090.stdout: osd default data pool replay window = 5 2021-02-20T08:05:29.409 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.409 INFO:teuthology.orchestra.run.smithi090.stdout: mon allow pool delete = true 2021-02-20T08:05:29.409 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.410 INFO:teuthology.orchestra.run.smithi090.stdout: mon cluster log file level = debug 2021-02-20T08:05:29.410 INFO:teuthology.orchestra.run.smithi090.stdout: debug asserts on shutdown = true 2021-02-20T08:05:29.410 INFO:teuthology.orchestra.run.smithi090.stdout: mon health detail to clog = false 2021-02-20T08:05:29.410 INFO:teuthology.orchestra.run.smithi090.stdout: mon host = 172.21.15.90 2021-02-20T08:05:29.410 INFO:teuthology.orchestra.run.smithi090.stdout: mon client directed command retry = 5 2021-02-20T08:05:29.411 INFO:teuthology.orchestra.run.smithi090.stdout: ms inject socket failures = 5000 2021-02-20T08:05:29.411 INFO:teuthology.orchestra.run.smithi090.stdout: fsid = 58fc22eb-10c4-4442-9a61-4d595829e8b5 2021-02-20T08:05:29.411 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.411 INFO:teuthology.orchestra.run.smithi090.stdout:[osd] 2021-02-20T08:05:29.412 INFO:teuthology.orchestra.run.smithi090.stdout: osd journal size = 100 2021-02-20T08:05:29.412 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.412 INFO:teuthology.orchestra.run.smithi090.stdout: osd scrub load threshold = 5.0 2021-02-20T08:05:29.412 INFO:teuthology.orchestra.run.smithi090.stdout: osd scrub max interval = 600 2021-02-20T08:05:29.412 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.413 INFO:teuthology.orchestra.run.smithi090.stdout: osd recover clone overlap = true 2021-02-20T08:05:29.413 INFO:teuthology.orchestra.run.smithi090.stdout: osd recovery max chunk = 1048576 2021-02-20T08:05:29.413 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.413 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug shutdown = true 2021-02-20T08:05:29.413 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug op order = true 2021-02-20T08:05:29.414 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug verify stray on activate = true 2021-02-20T08:05:29.414 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.414 INFO:teuthology.orchestra.run.smithi090.stdout: osd open classes on start = true 2021-02-20T08:05:29.414 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug pg log writeout = true 2021-02-20T08:05:29.414 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.415 INFO:teuthology.orchestra.run.smithi090.stdout: osd deep scrub update digest min age = 30 2021-02-20T08:05:29.415 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.415 INFO:teuthology.orchestra.run.smithi090.stdout: osd map max advance = 10 2021-02-20T08:05:29.415 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.415 INFO:teuthology.orchestra.run.smithi090.stdout: journal zero on create = true 2021-02-20T08:05:29.416 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.416 INFO:teuthology.orchestra.run.smithi090.stdout: filestore ondisk finisher threads = 3 2021-02-20T08:05:29.416 INFO:teuthology.orchestra.run.smithi090.stdout: filestore apply finisher threads = 3 2021-02-20T08:05:29.416 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.416 INFO:teuthology.orchestra.run.smithi090.stdout: bdev debug aio = true 2021-02-20T08:05:29.417 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug misdirected ops = true 2021-02-20T08:05:29.417 INFO:teuthology.orchestra.run.smithi090.stdout: bdev async discard = True 2021-02-20T08:05:29.417 INFO:teuthology.orchestra.run.smithi090.stdout: bdev enable discard = True 2021-02-20T08:05:29.417 INFO:teuthology.orchestra.run.smithi090.stdout: bluestore block size = 96636764160 2021-02-20T08:05:29.417 INFO:teuthology.orchestra.run.smithi090.stdout: bluestore fsck on mount = True 2021-02-20T08:05:29.418 INFO:teuthology.orchestra.run.smithi090.stdout: debug bluefs = 20 2021-02-20T08:05:29.418 INFO:teuthology.orchestra.run.smithi090.stdout: debug bluestore = 20 2021-02-20T08:05:29.418 INFO:teuthology.orchestra.run.smithi090.stdout: debug filestore = 20 2021-02-20T08:05:29.418 INFO:teuthology.orchestra.run.smithi090.stdout: debug journal = 20 2021-02-20T08:05:29.419 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 20 2021-02-20T08:05:29.419 INFO:teuthology.orchestra.run.smithi090.stdout: debug osd = 25 2021-02-20T08:05:29.419 INFO:teuthology.orchestra.run.smithi090.stdout: debug rocksdb = 10 2021-02-20T08:05:29.419 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd backfillfull_ratio = 0.85 2021-02-20T08:05:29.419 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd full ratio = 0.9 2021-02-20T08:05:29.420 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd nearfull ratio = 0.8 2021-02-20T08:05:29.420 INFO:teuthology.orchestra.run.smithi090.stdout: osd failsafe full ratio = 0.95 2021-02-20T08:05:29.420 INFO:teuthology.orchestra.run.smithi090.stdout: osd memory target = 1610612736 2021-02-20T08:05:29.420 INFO:teuthology.orchestra.run.smithi090.stdout: osd objectstore = bluestore 2021-02-20T08:05:29.420 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.421 INFO:teuthology.orchestra.run.smithi090.stdout:[mgr] 2021-02-20T08:05:29.421 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2021-02-20T08:05:29.421 INFO:teuthology.orchestra.run.smithi090.stdout: debug mgr = 20 2021-02-20T08:05:29.421 INFO:teuthology.orchestra.run.smithi090.stdout: debug mon = 20 2021-02-20T08:05:29.421 INFO:teuthology.orchestra.run.smithi090.stdout: debug auth = 20 2021-02-20T08:05:29.422 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min pgs per osd = 4 2021-02-20T08:05:29.422 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min bytes per osd = 10 2021-02-20T08:05:29.422 INFO:teuthology.orchestra.run.smithi090.stdout: mgr/telemetry/nag = false 2021-02-20T08:05:29.422 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.422 INFO:teuthology.orchestra.run.smithi090.stdout:[mon] 2021-02-20T08:05:29.423 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2021-02-20T08:05:29.423 INFO:teuthology.orchestra.run.smithi090.stdout: debug mon = 20 2021-02-20T08:05:29.423 INFO:teuthology.orchestra.run.smithi090.stdout: debug paxos = 20 2021-02-20T08:05:29.423 INFO:teuthology.orchestra.run.smithi090.stdout: debug auth = 20 2021-02-20T08:05:29.423 INFO:teuthology.orchestra.run.smithi090.stdout: mon data avail warn = 5 2021-02-20T08:05:29.424 INFO:teuthology.orchestra.run.smithi090.stdout: mon mgr mkfs grace = 240 2021-02-20T08:05:29.424 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min pgs per osd = 4 2021-02-20T08:05:29.424 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd reporter subtree level = osd 2021-02-20T08:05:29.424 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd prime pg temp = true 2021-02-20T08:05:29.425 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min bytes per osd = 10 2021-02-20T08:05:29.425 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:29.425 INFO:teuthology.orchestra.run.smithi090.stdout:[client] 2021-02-20T08:05:29.425 INFO:teuthology.orchestra.run.smithi090.stdout: rgw cache enabled = true 2021-02-20T08:05:29.425 INFO:teuthology.orchestra.run.smithi090.stdout: rgw enable ops log = true 2021-02-20T08:05:29.426 INFO:teuthology.orchestra.run.smithi090.stdout: rgw enable usage log = true 2021-02-20T08:05:29.426 INFO:teuthology.orchestra.run.smithi090.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-20T08:05:29.426 INFO:teuthology.orchestra.run.smithi090.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-20T08:05:29.426 INFO:teuthology.orchestra.run.smithi090.stdout: debug rbd = 20 2021-02-20T08:05:29.426 INFO:teuthology.orchestra.run.smithi090.stdout: debug rbd_mirror = 15 2021-02-20T08:05:29.427 INFO:teuthology.orchestra.run.smithi090.stdout: log to stderr = False 2021-02-20T08:05:29.427 INFO:teuthology.orchestra.run.smithi090.stdout: rbd default features = 125 2021-02-20T08:05:29.427 INFO:teuthology.orchestra.run.smithi090.stdout: rbd mirror image policy type = none 2021-02-20T08:05:29.427 INFO:teuthology.orchestra.run.smithi090.stdout:[mon.a] 2021-02-20T08:05:29.427 INFO:teuthology.orchestra.run.smithi090.stdout:[client.mirror.0] 2021-02-20T08:05:29.428 INFO:teuthology.orchestra.run.smithi090.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:05:29.428 INFO:teuthology.orchestra.run.smithi090.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:05:29.428 INFO:teuthology.orchestra.run.smithi090.stdout:[client.mirror.1] 2021-02-20T08:05:29.428 INFO:teuthology.orchestra.run.smithi090.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:05:29.428 INFO:teuthology.orchestra.run.smithi090.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:05:29.429 INFO:teuthology.orchestra.run.smithi090.stdout:[client.mirror.2] 2021-02-20T08:05:29.429 INFO:teuthology.orchestra.run.smithi090.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:05:29.429 INFO:teuthology.orchestra.run.smithi090.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:05:29.429 INFO:teuthology.orchestra.run.smithi090.stdout:[client.mirror.3] 2021-02-20T08:05:29.430 INFO:teuthology.orchestra.run.smithi090.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T08:05:29.430 INFO:teuthology.orchestra.run.smithi090.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T08:05:29.431 INFO:tasks.ceph:Creating admin key on cluster2.mon.a... 2021-02-20T08:05:29.432 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/cluster2.keyring 2021-02-20T08:05:29.520 INFO:tasks.ceph:Copying monmap to all nodes... 2021-02-20T08:05:29.521 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:29.521 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/cluster2.keyring of=/dev/stdout 2021-02-20T08:05:29.527 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:29.527 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/home/ubuntu/cephtest/cluster2.monmap of=/dev/stdout 2021-02-20T08:05:29.576 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi003.front.sepia.ceph.com 2021-02-20T08:05:29.576 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:05:29.577 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/cluster2.keyring 2021-02-20T08:05:29.577 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2021-02-20T08:05:29.603 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:05:29.604 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/cluster2.monmap 2021-02-20T08:05:29.653 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi090.front.sepia.ceph.com 2021-02-20T08:05:29.653 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:29.654 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/cluster2.keyring 2021-02-20T08:05:29.654 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2021-02-20T08:05:29.681 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:29.681 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/home/ubuntu/cephtest/cluster2.monmap 2021-02-20T08:05:29.728 INFO:tasks.ceph:Setting up mon nodes... 2021-02-20T08:05:29.728 INFO:tasks.ceph:Setting up mgr nodes... 2021-02-20T08:05:29.729 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mgr/cluster2-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/cluster2-x/keyring 2021-02-20T08:05:29.775 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mgr/cluster2-x/keyring 2021-02-20T08:05:29.777 INFO:tasks.ceph:Setting up mds nodes... 2021-02-20T08:05:29.777 INFO:tasks.ceph_client:Setting up client nodes... 2021-02-20T08:05:29.778 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/cluster2.client.0.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.0.keyring 2021-02-20T08:05:29.809 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/cluster2.client.0.keyring 2021-02-20T08:05:29.821 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror /etc/ceph/cluster2.client.mirror.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.keyring 2021-02-20T08:05:29.855 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/cluster2.client.mirror.keyring 2021-02-20T08:05:29.867 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.0 /etc/ceph/cluster2.client.mirror.0.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.0.keyring 2021-02-20T08:05:29.943 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/cluster2.client.mirror.0.keyring 2021-02-20T08:05:29.955 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.1 /etc/ceph/cluster2.client.mirror.1.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.1.keyring 2021-02-20T08:05:30.031 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/cluster2.client.mirror.1.keyring 2021-02-20T08:05:30.039 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.2 /etc/ceph/cluster2.client.mirror.2.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.2.keyring 2021-02-20T08:05:30.117 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/cluster2.client.mirror.2.keyring 2021-02-20T08:05:30.130 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.3 /etc/ceph/cluster2.client.mirror.3.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.3.keyring 2021-02-20T08:05:30.203 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/cluster2.client.mirror.3.keyring 2021-02-20T08:05:30.214 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.4 /etc/ceph/cluster2.client.mirror.4.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.4.keyring 2021-02-20T08:05:30.286 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/cluster2.client.mirror.4.keyring 2021-02-20T08:05:30.299 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.5 /etc/ceph/cluster2.client.mirror.5.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.5.keyring 2021-02-20T08:05:30.375 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/cluster2.client.mirror.5.keyring 2021-02-20T08:05:30.388 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.6 /etc/ceph/cluster2.client.mirror.6.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.6.keyring 2021-02-20T08:05:30.462 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/cluster2.client.mirror.6.keyring 2021-02-20T08:05:30.474 INFO:tasks.ceph:Running mkfs on osd nodes... 2021-02-20T08:05:30.475 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi090.front.sepia.ceph.com'): {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'}} 2021-02-20T08:05:30.475 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/osd/cluster2-0 2021-02-20T08:05:30.533 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T08:05:30.533 INFO:tasks.ceph:role: cluster2.osd.0 2021-02-20T08:05:30.534 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi090.front.sepia.ceph.com 2021-02-20T08:05:30.534 DEBUG:teuthology.orchestra.run.smithi090:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-02-20T08:05:31.049 INFO:teuthology.orchestra.run.smithi090.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2021-02-20T08:05:31.049 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-20T08:05:31.050 INFO:teuthology.orchestra.run.smithi090.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-02-20T08:05:31.050 INFO:teuthology.orchestra.run.smithi090.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-02-20T08:05:31.050 INFO:teuthology.orchestra.run.smithi090.stdout: = sunit=0 swidth=0 blks 2021-02-20T08:05:31.050 INFO:teuthology.orchestra.run.smithi090.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-20T08:05:31.051 INFO:teuthology.orchestra.run.smithi090.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-02-20T08:05:31.051 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-20T08:05:31.051 INFO:teuthology.orchestra.run.smithi090.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-20T08:05:31.052 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi090.front.sepia.ceph.com -o noatime 2021-02-20T08:05:31.052 DEBUG:teuthology.orchestra.run.smithi090:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/cluster2-0 2021-02-20T08:05:31.128 DEBUG:teuthology.orchestra.run.smithi090:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-0 2021-02-20T08:05:31.139 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: /sbin/restorecon: command not found 2021-02-20T08:05:31.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:05:31.140 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/osd/cluster2-1 2021-02-20T08:05:31.196 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T08:05:31.196 INFO:tasks.ceph:role: cluster2.osd.1 2021-02-20T08:05:31.196 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi090.front.sepia.ceph.com 2021-02-20T08:05:31.197 DEBUG:teuthology.orchestra.run.smithi090:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-02-20T08:05:31.706 INFO:teuthology.orchestra.run.smithi090.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2021-02-20T08:05:31.706 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-20T08:05:31.707 INFO:teuthology.orchestra.run.smithi090.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-02-20T08:05:31.707 INFO:teuthology.orchestra.run.smithi090.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-02-20T08:05:31.707 INFO:teuthology.orchestra.run.smithi090.stdout: = sunit=0 swidth=0 blks 2021-02-20T08:05:31.707 INFO:teuthology.orchestra.run.smithi090.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-20T08:05:31.708 INFO:teuthology.orchestra.run.smithi090.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-02-20T08:05:31.708 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-20T08:05:31.708 INFO:teuthology.orchestra.run.smithi090.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-20T08:05:31.709 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi090.front.sepia.ceph.com -o noatime 2021-02-20T08:05:31.710 DEBUG:teuthology.orchestra.run.smithi090:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/cluster2-1 2021-02-20T08:05:31.789 DEBUG:teuthology.orchestra.run.smithi090:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-1 2021-02-20T08:05:31.801 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: /sbin/restorecon: command not found 2021-02-20T08:05:31.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:05:31.803 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/osd/cluster2-2 2021-02-20T08:05:31.861 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T08:05:31.861 INFO:tasks.ceph:role: cluster2.osd.2 2021-02-20T08:05:31.862 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi090.front.sepia.ceph.com 2021-02-20T08:05:31.862 DEBUG:teuthology.orchestra.run.smithi090:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-02-20T08:05:32.371 INFO:teuthology.orchestra.run.smithi090.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2021-02-20T08:05:32.371 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-20T08:05:32.371 INFO:teuthology.orchestra.run.smithi090.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-02-20T08:05:32.372 INFO:teuthology.orchestra.run.smithi090.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-02-20T08:05:32.372 INFO:teuthology.orchestra.run.smithi090.stdout: = sunit=0 swidth=0 blks 2021-02-20T08:05:32.372 INFO:teuthology.orchestra.run.smithi090.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-20T08:05:32.372 INFO:teuthology.orchestra.run.smithi090.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-02-20T08:05:32.372 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-20T08:05:32.373 INFO:teuthology.orchestra.run.smithi090.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-20T08:05:32.374 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi090.front.sepia.ceph.com -o noatime 2021-02-20T08:05:32.374 DEBUG:teuthology.orchestra.run.smithi090:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/cluster2-2 2021-02-20T08:05:32.426 DEBUG:teuthology.orchestra.run.smithi090:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-2 2021-02-20T08:05:32.438 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: /sbin/restorecon: command not found 2021-02-20T08:05:32.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:05:32.439 DEBUG:teuthology.orchestra.run.smithi090:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/cluster2.monmap 2021-02-20T08:05:32.536 INFO:teuthology.orchestra.run.smithi090.stderr:2021-02-20T08:05:32.535+0000 7f419a99a080 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-0/keyring: can't open /var/lib/ceph/osd/cluster2-0/keyring: (2) No such file or directory 2021-02-20T08:05:32.537 INFO:teuthology.orchestra.run.smithi090.stderr:2021-02-20T08:05:32.535+0000 7f419a99a080 -1 created new key in keyring /var/lib/ceph/osd/cluster2-0/keyring 2021-02-20T08:05:32.538 INFO:teuthology.orchestra.run.smithi090.stderr:2021-02-20T08:05:32.535+0000 7f419a99a080 -1 bluestore(/var/lib/ceph/osd/cluster2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-0/block: (2) No such file or directory 2021-02-20T08:05:32.539 INFO:teuthology.orchestra.run.smithi090.stderr:2021-02-20T08:05:32.535+0000 7f419a99a080 -1 bluestore(/var/lib/ceph/osd/cluster2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-0/block: (2) No such file or directory 2021-02-20T08:05:32.539 INFO:teuthology.orchestra.run.smithi090.stderr:2021-02-20T08:05:32.535+0000 7f419a99a080 -1 bluestore(/var/lib/ceph/osd/cluster2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-0/block: (2) No such file or directory 2021-02-20T08:05:32.542 INFO:teuthology.orchestra.run.smithi090.stderr:2021-02-20T08:05:32.543+0000 7f419a99a080 -1 bluestore(/var/lib/ceph/osd/cluster2-0) _read_fsid unparsable uuid 2021-02-20T08:05:35.507 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-0 2021-02-20T08:05:35.523 DEBUG:teuthology.orchestra.run.smithi090:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/cluster2.monmap 2021-02-20T08:05:35.604 INFO:teuthology.orchestra.run.smithi090.stderr:2021-02-20T08:05:35.603+0000 7f6eb5724080 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-1/keyring: can't open /var/lib/ceph/osd/cluster2-1/keyring: (2) No such file or directory 2021-02-20T08:05:35.604 INFO:teuthology.orchestra.run.smithi090.stderr:2021-02-20T08:05:35.603+0000 7f6eb5724080 -1 created new key in keyring /var/lib/ceph/osd/cluster2-1/keyring 2021-02-20T08:05:35.605 INFO:teuthology.orchestra.run.smithi090.stderr:2021-02-20T08:05:35.603+0000 7f6eb5724080 -1 bluestore(/var/lib/ceph/osd/cluster2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-1/block: (2) No such file or directory 2021-02-20T08:05:35.605 INFO:teuthology.orchestra.run.smithi090.stderr:2021-02-20T08:05:35.603+0000 7f6eb5724080 -1 bluestore(/var/lib/ceph/osd/cluster2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-1/block: (2) No such file or directory 2021-02-20T08:05:35.605 INFO:teuthology.orchestra.run.smithi090.stderr:2021-02-20T08:05:35.603+0000 7f6eb5724080 -1 bluestore(/var/lib/ceph/osd/cluster2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-1/block: (2) No such file or directory 2021-02-20T08:05:35.606 INFO:teuthology.orchestra.run.smithi090.stderr:2021-02-20T08:05:35.603+0000 7f6eb5724080 -1 bluestore(/var/lib/ceph/osd/cluster2-1) _read_fsid unparsable uuid 2021-02-20T08:05:38.554 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-1 2021-02-20T08:05:38.613 DEBUG:teuthology.orchestra.run.smithi090:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/cluster2.monmap 2021-02-20T08:05:38.697 INFO:teuthology.orchestra.run.smithi090.stderr:2021-02-20T08:05:38.691+0000 7f981e366080 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-2/keyring: can't open /var/lib/ceph/osd/cluster2-2/keyring: (2) No such file or directory 2021-02-20T08:05:38.719 INFO:teuthology.orchestra.run.smithi090.stderr:2021-02-20T08:05:38.691+0000 7f981e366080 -1 created new key in keyring /var/lib/ceph/osd/cluster2-2/keyring 2021-02-20T08:05:38.719 INFO:teuthology.orchestra.run.smithi090.stderr:2021-02-20T08:05:38.691+0000 7f981e366080 -1 bluestore(/var/lib/ceph/osd/cluster2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-2/block: (2) No such file or directory 2021-02-20T08:05:38.720 INFO:teuthology.orchestra.run.smithi090.stderr:2021-02-20T08:05:38.691+0000 7f981e366080 -1 bluestore(/var/lib/ceph/osd/cluster2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-2/block: (2) No such file or directory 2021-02-20T08:05:38.721 INFO:teuthology.orchestra.run.smithi090.stderr:2021-02-20T08:05:38.691+0000 7f981e366080 -1 bluestore(/var/lib/ceph/osd/cluster2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-2/block: (2) No such file or directory 2021-02-20T08:05:38.721 INFO:teuthology.orchestra.run.smithi090.stderr:2021-02-20T08:05:38.691+0000 7f981e366080 -1 bluestore(/var/lib/ceph/osd/cluster2-2) _read_fsid unparsable uuid 2021-02-20T08:05:41.666 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-2 2021-02-20T08:05:41.725 INFO:tasks.ceph:Reading keys from all nodes... 2021-02-20T08:05:41.726 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:05:41.727 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mgr/cluster2-x/keyring of=/dev/stdout 2021-02-20T08:05:41.744 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:41.744 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/cluster2-0/keyring of=/dev/stdout 2021-02-20T08:05:41.782 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:41.782 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/cluster2-1/keyring of=/dev/stdout 2021-02-20T08:05:41.840 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:41.840 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/cluster2-2/keyring of=/dev/stdout 2021-02-20T08:05:41.894 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2021-02-20T08:05:41.895 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/cluster2.client.0.keyring of=/dev/stdout 2021-02-20T08:05:41.901 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:41.902 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/cluster2.client.mirror.keyring of=/dev/stdout 2021-02-20T08:05:41.943 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:41.944 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/cluster2.client.mirror.0.keyring of=/dev/stdout 2021-02-20T08:05:41.992 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:41.992 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/cluster2.client.mirror.1.keyring of=/dev/stdout 2021-02-20T08:05:42.040 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:42.040 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/cluster2.client.mirror.2.keyring of=/dev/stdout 2021-02-20T08:05:42.088 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:42.088 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/cluster2.client.mirror.3.keyring of=/dev/stdout 2021-02-20T08:05:42.135 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:42.136 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/cluster2.client.mirror.4.keyring of=/dev/stdout 2021-02-20T08:05:42.183 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:42.184 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/cluster2.client.mirror.5.keyring of=/dev/stdout 2021-02-20T08:05:42.232 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:42.233 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/cluster2.client.mirror.6.keyring of=/dev/stdout 2021-02-20T08:05:42.280 INFO:tasks.ceph:Adding keys to all mons... 2021-02-20T08:05:42.280 DEBUG:teuthology.orchestra.run.smithi090:> sudo tee -a /etc/ceph/cluster2.keyring 2021-02-20T08:05:42.335 INFO:teuthology.orchestra.run.smithi090.stdout:[mgr.x] 2021-02-20T08:05:42.335 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQDJwjBggE1JLhAA1tvvwY0uVT2ftqLPRocoGA== 2021-02-20T08:05:42.336 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.0] 2021-02-20T08:05:42.336 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQDMwjBghjYdIBAA26+xXfeNZjW2hzRwg8vqDw== 2021-02-20T08:05:42.336 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.1] 2021-02-20T08:05:42.336 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQDPwjBg9tEgJBAAAbNUaHZ7VMI3gFeY9WehVg== 2021-02-20T08:05:42.336 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.2] 2021-02-20T08:05:42.337 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQDSwjBgajphKRAAZ5OLbaLcG9p5qyP+KTrdIA== 2021-02-20T08:05:42.337 INFO:teuthology.orchestra.run.smithi090.stdout:[client.0] 2021-02-20T08:05:42.337 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQDJwjBgBE5PMBAAhhv42puD/13PR6v944zMTQ== 2021-02-20T08:05:42.337 INFO:teuthology.orchestra.run.smithi090.stdout:[client.mirror] 2021-02-20T08:05:42.338 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQDJwjBglCIdMxAACObKryJB9MLGM8cLQ3U7tQ== 2021-02-20T08:05:42.338 INFO:teuthology.orchestra.run.smithi090.stdout:[client.mirror.0] 2021-02-20T08:05:42.338 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQDJwjBgn2pbOBAAwaOLUdyLjI8OzZN2K98rsg== 2021-02-20T08:05:42.338 INFO:teuthology.orchestra.run.smithi090.stdout:[client.mirror.1] 2021-02-20T08:05:42.339 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQDKwjBgFCz9ARAAtar1dF3uDj77ku8wh4VySg== 2021-02-20T08:05:42.339 INFO:teuthology.orchestra.run.smithi090.stdout:[client.mirror.2] 2021-02-20T08:05:42.339 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQDKwjBgDgEkBxAAUOncSoi8ZJuBiWoxjsubaw== 2021-02-20T08:05:42.340 INFO:teuthology.orchestra.run.smithi090.stdout:[client.mirror.3] 2021-02-20T08:05:42.340 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQDKwjBgBwM7DBAAqVQsWKkiVLOvk+XaXKeazQ== 2021-02-20T08:05:42.340 INFO:teuthology.orchestra.run.smithi090.stdout:[client.mirror.4] 2021-02-20T08:05:42.340 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQDKwjBgcIgzERAALxBZv9z5LAS7B13EKKiSyQ== 2021-02-20T08:05:42.340 INFO:teuthology.orchestra.run.smithi090.stdout:[client.mirror.5] 2021-02-20T08:05:42.341 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQDKwjBgTNd4FhAARHlwZq5eNqCphWfWRHot4Q== 2021-02-20T08:05:42.341 INFO:teuthology.orchestra.run.smithi090.stdout:[client.mirror.6] 2021-02-20T08:05:42.341 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQDKwjBgLVi4GxAAp/VQU50mKWq7azXMHMichg== 2021-02-20T08:05:42.342 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-02-20T08:05:42.427 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-20T08:05:42.513 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-20T08:05:42.596 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-20T08:05:42.680 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T08:05:42.767 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T08:05:42.807 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T08:05:42.897 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T08:05:42.983 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T08:05:43.071 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T08:05:43.158 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.4 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T08:05:43.248 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.5 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T08:05:43.293 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.6 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T08:05:43.379 INFO:tasks.ceph:Running mkfs on mon nodes... 2021-02-20T08:05:43.379 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/mon/cluster2-a 2021-02-20T08:05:43.436 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster cluster2 --mkfs -i a --monmap /home/ubuntu/cephtest/cluster2.monmap --keyring /etc/ceph/cluster2.keyring 2021-02-20T08:05:43.549 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/mon/cluster2-a 2021-02-20T08:05:43.564 DEBUG:teuthology.orchestra.run.smithi090:> rm -- /home/ubuntu/cephtest/cluster2.monmap 2021-02-20T08:05:43.611 INFO:tasks.ceph:Starting mon daemons in cluster cluster2... 2021-02-20T08:05:43.612 INFO:tasks.ceph.cluster2.mon.a:Restarting daemon 2021-02-20T08:05:43.612 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster cluster2 -i a 2021-02-20T08:05:43.656 INFO:tasks.ceph.cluster2.mon.a:Started 2021-02-20T08:05:43.656 INFO:tasks.ceph:Starting mgr daemons in cluster cluster2... 2021-02-20T08:05:43.657 INFO:tasks.ceph.cluster2.mgr.x:Restarting daemon 2021-02-20T08:05:43.658 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster cluster2 -i x 2021-02-20T08:05:43.664 INFO:tasks.ceph.cluster2.mgr.x:Started 2021-02-20T08:05:43.665 INFO:tasks.ceph:Setting crush tunables to default 2021-02-20T08:05:43.665 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster cluster2 osd crush tunables default 2021-02-20T08:05:44.014 INFO:teuthology.orchestra.run.smithi090.stderr:adjusted tunables profile to default 2021-02-20T08:05:44.027 INFO:tasks.ceph:Starting osd daemons in cluster cluster2... 2021-02-20T08:05:44.027 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:44.028 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/cluster2-0/fsid of=/dev/stdout 2021-02-20T08:05:44.078 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:44.078 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/cluster2-1/fsid of=/dev/stdout 2021-02-20T08:05:44.158 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:05:44.158 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/cluster2-2/fsid of=/dev/stdout 2021-02-20T08:05:44.237 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster cluster2 osd new 2d81d853-654d-4d47-bbe6-42c0ae14fc03 0 2021-02-20T08:05:44.534 INFO:teuthology.orchestra.run.smithi090.stdout:0 2021-02-20T08:05:44.548 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster cluster2 osd new 52db12e2-2183-4718-9a85-bacc336baaa6 1 2021-02-20T08:05:44.767 INFO:teuthology.orchestra.run.smithi090.stdout:1 2021-02-20T08:05:44.782 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster cluster2 osd new f1711601-00cf-476d-aace-51bb0388c323 2 2021-02-20T08:05:45.006 INFO:teuthology.orchestra.run.smithi090.stdout:2 2021-02-20T08:05:45.020 INFO:tasks.ceph.cluster2.osd.0:Restarting daemon 2021-02-20T08:05:45.020 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 0 2021-02-20T08:05:45.026 INFO:tasks.ceph.cluster2.osd.0:Started 2021-02-20T08:05:45.026 INFO:tasks.ceph.cluster2.osd.1:Restarting daemon 2021-02-20T08:05:45.026 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 1 2021-02-20T08:05:45.030 INFO:tasks.ceph.cluster2.osd.1:Started 2021-02-20T08:05:45.031 INFO:tasks.ceph.cluster2.osd.2:Restarting daemon 2021-02-20T08:05:45.031 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 2 2021-02-20T08:05:45.035 INFO:tasks.ceph.cluster2.osd.2:Started 2021-02-20T08:05:45.036 INFO:tasks.ceph:Waiting for OSDs to come up 2021-02-20T08:05:45.340 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster2 osd dump --format=json 2021-02-20T08:05:45.548 INFO:teuthology.misc.health.smithi090.stdout: 2021-02-20T08:05:45.548 INFO:teuthology.misc.health.smithi090.stdout:{"epoch":5,"fsid":"58fc22eb-10c4-4442-9a61-4d595829e8b5","created":"2021-02-20T08:05:43.784088+0000","modified":"2021-02-20T08:05:45.006108+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"2d81d853-654d-4d47-bbe6-42c0ae14fc03","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"52db12e2-2183-4718-9a85-bacc336baaa6","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f1711601-00cf-476d-aace-51bb0388c323","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T08:05:45.560 DEBUG:teuthology.misc:0 of 3 OSDs are up 2021-02-20T08:05:45.652 INFO:tasks.ceph.cluster2.osd.1.smithi090.stderr:2021-02-20T08:05:45.651+0000 7f3906a0a080 -1 Falling back to public interface 2021-02-20T08:05:45.653 INFO:tasks.ceph.cluster2.osd.0.smithi090.stderr:2021-02-20T08:05:45.651+0000 7f7be93b1080 -1 Falling back to public interface 2021-02-20T08:05:45.653 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T08:05:45.651+0000 7f731fbed080 -1 Falling back to public interface 2021-02-20T08:05:49.141 INFO:tasks.ceph.cluster2.osd.0.smithi090.stderr:2021-02-20T08:05:49.139+0000 7f7be93b1080 -1 osd.0 0 log_to_monitors {default=true} 2021-02-20T08:05:49.334 INFO:tasks.ceph.cluster2.osd.1.smithi090.stderr:2021-02-20T08:05:49.331+0000 7f3906a0a080 -1 osd.1 0 log_to_monitors {default=true} 2021-02-20T08:05:49.436 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T08:05:49.435+0000 7f731fbed080 -1 osd.2 0 log_to_monitors {default=true} 2021-02-20T08:05:51.068 INFO:tasks.ceph.cluster2.osd.0.smithi090.stderr:2021-02-20T08:05:51.067+0000 7f7be087a700 -1 osd.0 0 waiting for initial osdmap 2021-02-20T08:05:51.069 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T08:05:51.067+0000 7f73170b6700 -1 osd.2 0 waiting for initial osdmap 2021-02-20T08:05:51.071 INFO:tasks.ceph.cluster2.osd.1.smithi090.stderr:2021-02-20T08:05:51.071+0000 7f38ff6d6700 -1 osd.1 0 waiting for initial osdmap 2021-02-20T08:05:51.865 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster2 osd dump --format=json 2021-02-20T08:05:52.169 INFO:teuthology.misc.health.smithi090.stdout: 2021-02-20T08:05:52.169 INFO:teuthology.misc.health.smithi090.stdout:{"epoch":8,"fsid":"58fc22eb-10c4-4442-9a61-4d595829e8b5","created":"2021-02-20T08:05:43.784088+0000","modified":"2021-02-20T08:05:52.061316+0000","last_up_change":"2021-02-20T08:05:52.061316+0000","last_in_change":"2021-02-20T08:05:52.061316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"2d81d853-654d-4d47-bbe6-42c0ae14fc03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":15695},{"type":"v1","addr":"172.21.15.90:6808","nonce":15695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6811","nonce":15695},{"type":"v1","addr":"172.21.15.90:6815","nonce":15695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":15695},{"type":"v1","addr":"172.21.15.90:6822","nonce":15695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":15695},{"type":"v1","addr":"172.21.15.90:6818","nonce":15695}]},"public_addr":"172.21.15.90:6808/15695","cluster_addr":"172.21.15.90:6815/15695","heartbeat_back_addr":"172.21.15.90:6822/15695","heartbeat_front_addr":"172.21.15.90:6818/15695","state":["exists","up"]},{"osd":1,"uuid":"52db12e2-2183-4718-9a85-bacc336baaa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":15697},{"type":"v1","addr":"172.21.15.90:6801","nonce":15697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":15697},{"type":"v1","addr":"172.21.15.90:6803","nonce":15697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":15697},{"type":"v1","addr":"172.21.15.90:6813","nonce":15697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":15697},{"type":"v1","addr":"172.21.15.90:6807","nonce":15697}]},"public_addr":"172.21.15.90:6801/15697","cluster_addr":"172.21.15.90:6803/15697","heartbeat_back_addr":"172.21.15.90:6813/15697","heartbeat_front_addr":"172.21.15.90:6807/15697","state":["exists","up"]},{"osd":2,"uuid":"f1711601-00cf-476d-aace-51bb0388c323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":15696},{"type":"v1","addr":"172.21.15.90:6809","nonce":15696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":15696},{"type":"v1","addr":"172.21.15.90:6814","nonce":15696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":15696},{"type":"v1","addr":"172.21.15.90:6823","nonce":15696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":15696},{"type":"v1","addr":"172.21.15.90:6819","nonce":15696}]},"public_addr":"172.21.15.90:6809/15696","cluster_addr":"172.21.15.90:6814/15696","heartbeat_back_addr":"172.21.15.90:6823/15696","heartbeat_front_addr":"172.21.15.90:6819/15696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T08:05:52.188 DEBUG:teuthology.misc:3 of 3 OSDs are up 2021-02-20T08:05:52.188 INFO:tasks.ceph:Creating RBD pool 2021-02-20T08:05:52.189 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster cluster2 osd pool create rbd 8 2021-02-20T08:05:53.088 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'rbd' created 2021-02-20T08:05:53.104 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster cluster2 osd pool application enable rbd rbd --yes-i-really-mean-it 2021-02-20T08:05:54.102 INFO:teuthology.orchestra.run.smithi090.stderr:enabled application 'rbd' on pool 'rbd' 2021-02-20T08:05:54.115 INFO:tasks.ceph:Starting mds daemons in cluster cluster2... 2021-02-20T08:05:54.115 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2021-02-20T08:05:54.116 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2021-02-20T08:05:54.406 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:54.406 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":10,"fsid":"58fc22eb-10c4-4442-9a61-4d595829e8b5","created":"2021-02-20T08:05:43.784088+0000","modified":"2021-02-20T08:05:54.080217+0000","last_up_change":"2021-02-20T08:05:52.061316+0000","last_in_change":"2021-02-20T08:05:52.061316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-02-20T08:05:52.131492+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-02-20T08:05:52.547051+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"2d81d853-654d-4d47-bbe6-42c0ae14fc03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":15695},{"type":"v1","addr":"172.21.15.90:6808","nonce":15695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6811","nonce":15695},{"type":"v1","addr":"172.21.15.90:6815","nonce":15695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":15695},{"type":"v1","addr":"172.21.15.90:6822","nonce":15695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":15695},{"type":"v1","addr":"172.21.15.90:6818","nonce":15695}]},"public_addr":"172.21.15.90:6808/15695","cluster_addr":"172.21.15.90:6815/15695","heartbeat_back_addr":"172.21.15.90:6822/15695","heartbeat_front_addr":"172.21.15.90:6818/15695","state":["exists","up"]},{"osd":1,"uuid":"52db12e2-2183-4718-9a85-bacc336baaa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":15697},{"type":"v1","addr":"172.21.15.90:6801","nonce":15697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":15697},{"type":"v1","addr":"172.21.15.90:6803","nonce":15697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":15697},{"type":"v1","addr":"172.21.15.90:6813","nonce":15697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":15697},{"type":"v1","addr":"172.21.15.90:6807","nonce":15697}]},"public_addr":"172.21.15.90:6801/15697","cluster_addr":"172.21.15.90:6803/15697","heartbeat_back_addr":"172.21.15.90:6813/15697","heartbeat_front_addr":"172.21.15.90:6807/15697","state":["exists","up"]},{"osd":2,"uuid":"f1711601-00cf-476d-aace-51bb0388c323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":15696},{"type":"v1","addr":"172.21.15.90:6809","nonce":15696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":15696},{"type":"v1","addr":"172.21.15.90:6814","nonce":15696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":15696},{"type":"v1","addr":"172.21.15.90:6823","nonce":15696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":15696},{"type":"v1","addr":"172.21.15.90:6819","nonce":15696}]},"public_addr":"172.21.15.90:6809/15696","cluster_addr":"172.21.15.90:6814/15696","heartbeat_back_addr":"172.21.15.90:6823/15696","heartbeat_front_addr":"172.21.15.90:6819/15696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T08:05:50.122577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T08:05:50.351764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T08:05:50.400286+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T08:05:54.420 INFO:tasks.ceph.ceph_manager.cluster2:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-02-20T08:05:52.131492+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-02-20T08:05:52.547051+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-20T08:05:54.421 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd pool get device_health_metrics pg_num 2021-02-20T08:05:54.721 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2021-02-20T08:05:54.736 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd pool get rbd pg_num 2021-02-20T08:05:55.043 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 8 2021-02-20T08:05:55.058 INFO:tasks.ceph:Waiting until cluster2 daemons up and pgs clean... 2021-02-20T08:05:55.058 INFO:tasks.ceph.ceph_manager.cluster2:waiting for mgr available 2021-02-20T08:05:55.059 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 mgr dump --format=json 2021-02-20T08:05:55.397 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:55.411 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":4,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":18822},{"type":"v1","addr":"172.21.15.3:6827","nonce":18822}]},"active_addr":"172.21.15.3:6827/18822","active_change":"2021-02-20T08:05:47.991016+0000","active_mgr_features":4540138303579357183,"available":true,"standbys":[],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"haproxy","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":2396135839}]},{"addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":1231769756}]}]}} 2021-02-20T08:05:55.414 INFO:tasks.ceph.ceph_manager.cluster2:mgr available! 2021-02-20T08:05:55.414 INFO:tasks.ceph.ceph_manager.cluster2:waiting for all up 2021-02-20T08:05:55.415 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2021-02-20T08:05:55.711 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:55.711 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":11,"fsid":"58fc22eb-10c4-4442-9a61-4d595829e8b5","created":"2021-02-20T08:05:43.784088+0000","modified":"2021-02-20T08:05:55.093919+0000","last_up_change":"2021-02-20T08:05:52.061316+0000","last_in_change":"2021-02-20T08:05:52.061316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-02-20T08:05:52.131492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-02-20T08:05:52.547051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"2d81d853-654d-4d47-bbe6-42c0ae14fc03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":15695},{"type":"v1","addr":"172.21.15.90:6808","nonce":15695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6811","nonce":15695},{"type":"v1","addr":"172.21.15.90:6815","nonce":15695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":15695},{"type":"v1","addr":"172.21.15.90:6822","nonce":15695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":15695},{"type":"v1","addr":"172.21.15.90:6818","nonce":15695}]},"public_addr":"172.21.15.90:6808/15695","cluster_addr":"172.21.15.90:6815/15695","heartbeat_back_addr":"172.21.15.90:6822/15695","heartbeat_front_addr":"172.21.15.90:6818/15695","state":["exists","up"]},{"osd":1,"uuid":"52db12e2-2183-4718-9a85-bacc336baaa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":15697},{"type":"v1","addr":"172.21.15.90:6801","nonce":15697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":15697},{"type":"v1","addr":"172.21.15.90:6803","nonce":15697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":15697},{"type":"v1","addr":"172.21.15.90:6813","nonce":15697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":15697},{"type":"v1","addr":"172.21.15.90:6807","nonce":15697}]},"public_addr":"172.21.15.90:6801/15697","cluster_addr":"172.21.15.90:6803/15697","heartbeat_back_addr":"172.21.15.90:6813/15697","heartbeat_front_addr":"172.21.15.90:6807/15697","state":["exists","up"]},{"osd":2,"uuid":"f1711601-00cf-476d-aace-51bb0388c323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":15696},{"type":"v1","addr":"172.21.15.90:6809","nonce":15696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":15696},{"type":"v1","addr":"172.21.15.90:6814","nonce":15696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":15696},{"type":"v1","addr":"172.21.15.90:6823","nonce":15696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":15696},{"type":"v1","addr":"172.21.15.90:6819","nonce":15696}]},"public_addr":"172.21.15.90:6809/15696","cluster_addr":"172.21.15.90:6814/15696","heartbeat_back_addr":"172.21.15.90:6823/15696","heartbeat_front_addr":"172.21.15.90:6819/15696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T08:05:50.122577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T08:05:50.351764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T08:05:50.400286+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T08:05:55.725 INFO:tasks.ceph.ceph_manager.cluster2:all up! 2021-02-20T08:05:55.726 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2021-02-20T08:05:56.031 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:56.032 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":11,"fsid":"58fc22eb-10c4-4442-9a61-4d595829e8b5","created":"2021-02-20T08:05:43.784088+0000","modified":"2021-02-20T08:05:55.093919+0000","last_up_change":"2021-02-20T08:05:52.061316+0000","last_in_change":"2021-02-20T08:05:52.061316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-02-20T08:05:52.131492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-02-20T08:05:52.547051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"2d81d853-654d-4d47-bbe6-42c0ae14fc03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":15695},{"type":"v1","addr":"172.21.15.90:6808","nonce":15695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6811","nonce":15695},{"type":"v1","addr":"172.21.15.90:6815","nonce":15695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":15695},{"type":"v1","addr":"172.21.15.90:6822","nonce":15695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":15695},{"type":"v1","addr":"172.21.15.90:6818","nonce":15695}]},"public_addr":"172.21.15.90:6808/15695","cluster_addr":"172.21.15.90:6815/15695","heartbeat_back_addr":"172.21.15.90:6822/15695","heartbeat_front_addr":"172.21.15.90:6818/15695","state":["exists","up"]},{"osd":1,"uuid":"52db12e2-2183-4718-9a85-bacc336baaa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":15697},{"type":"v1","addr":"172.21.15.90:6801","nonce":15697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":15697},{"type":"v1","addr":"172.21.15.90:6803","nonce":15697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":15697},{"type":"v1","addr":"172.21.15.90:6813","nonce":15697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":15697},{"type":"v1","addr":"172.21.15.90:6807","nonce":15697}]},"public_addr":"172.21.15.90:6801/15697","cluster_addr":"172.21.15.90:6803/15697","heartbeat_back_addr":"172.21.15.90:6813/15697","heartbeat_front_addr":"172.21.15.90:6807/15697","state":["exists","up"]},{"osd":2,"uuid":"f1711601-00cf-476d-aace-51bb0388c323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":15696},{"type":"v1","addr":"172.21.15.90:6809","nonce":15696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":15696},{"type":"v1","addr":"172.21.15.90:6814","nonce":15696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":15696},{"type":"v1","addr":"172.21.15.90:6823","nonce":15696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":15696},{"type":"v1","addr":"172.21.15.90:6819","nonce":15696}]},"public_addr":"172.21.15.90:6809/15696","cluster_addr":"172.21.15.90:6814/15696","heartbeat_back_addr":"172.21.15.90:6823/15696","heartbeat_front_addr":"172.21.15.90:6819/15696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T08:05:50.122577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T08:05:50.351764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T08:05:50.400286+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T08:05:56.046 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.0 flush_pg_stats 2021-02-20T08:05:56.220 INFO:teuthology.orchestra.run.smithi090.stdout:34359738370 2021-02-20T08:05:56.220 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.1 flush_pg_stats 2021-02-20T08:05:56.395 INFO:teuthology.orchestra.run.smithi090.stdout:34359738370 2021-02-20T08:05:56.396 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.2 flush_pg_stats 2021-02-20T08:05:56.567 INFO:teuthology.orchestra.run.smithi090.stdout:34359738371 2021-02-20T08:05:56.568 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.0 2021-02-20T08:05:56.867 INFO:teuthology.orchestra.run.smithi090.stdout:34359738369 2021-02-20T08:05:56.881 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738370 got 34359738369 for osd.0 2021-02-20T08:05:57.882 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.0 2021-02-20T08:05:58.193 INFO:teuthology.orchestra.run.smithi090.stdout:34359738370 2021-02-20T08:05:58.208 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738370 got 34359738370 for osd.0 2021-02-20T08:05:58.208 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.1 2021-02-20T08:05:58.498 INFO:teuthology.orchestra.run.smithi090.stdout:34359738371 2021-02-20T08:05:58.513 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738370 got 34359738371 for osd.1 2021-02-20T08:05:58.513 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.2 2021-02-20T08:05:58.810 INFO:teuthology.orchestra.run.smithi090.stdout:34359738372 2021-02-20T08:05:58.825 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738371 got 34359738372 for osd.2 2021-02-20T08:05:58.826 INFO:tasks.ceph.ceph_manager.cluster2:waiting for clean 2021-02-20T08:05:58.826 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2021-02-20T08:05:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:59.124 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2021-02-20T08:05:59.135 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2021-02-20T08:05:58.000563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":14632,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":283100888,"statfs":{"total":289910292480,"available":289895309312,"internally_reserved":0,"allocated":278528,"data_stored":72292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:54.108806+0000","last_change":"2021-02-20T08:05:54.104973+0000","last_active":"2021-02-20T08:05:54.108806+0000","last_peered":"2021-02-20T08:05:54.108806+0000","last_clean":"2021-02-20T08:05:54.108806+0000","last_became_active":"2021-02-20T08:05:54.104618+0000","last_became_peered":"2021-02-20T08:05:54.104618+0000","last_unstale":"2021-02-20T08:05:54.108806+0000","last_undegraded":"2021-02-20T08:05:54.108806+0000","last_fullsized":"2021-02-20T08:05:54.108806+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_clean_scrub_stamp":"2021-02-20T08:05:53.070215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:54.108796+0000","last_change":"2021-02-20T08:05:54.104301+0000","last_active":"2021-02-20T08:05:54.108796+0000","last_peered":"2021-02-20T08:05:54.108796+0000","last_clean":"2021-02-20T08:05:54.108796+0000","last_became_active":"2021-02-20T08:05:54.103913+0000","last_became_peered":"2021-02-20T08:05:54.103913+0000","last_unstale":"2021-02-20T08:05:54.108796+0000","last_undegraded":"2021-02-20T08:05:54.108796+0000","last_fullsized":"2021-02-20T08:05:54.108796+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_clean_scrub_stamp":"2021-02-20T08:05:53.070215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:54.105899+0000","last_change":"2021-02-20T08:05:54.104028+0000","last_active":"2021-02-20T08:05:54.105899+0000","last_peered":"2021-02-20T08:05:54.105899+0000","last_clean":"2021-02-20T08:05:54.105899+0000","last_became_active":"2021-02-20T08:05:54.103735+0000","last_became_peered":"2021-02-20T08:05:54.103735+0000","last_unstale":"2021-02-20T08:05:54.105899+0000","last_undegraded":"2021-02-20T08:05:54.105899+0000","last_fullsized":"2021-02-20T08:05:54.105899+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_clean_scrub_stamp":"2021-02-20T08:05:53.070215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:54.099811+0000","last_change":"2021-02-20T08:05:54.099811+0000","last_active":"2021-02-20T08:05:54.099811+0000","last_peered":"2021-02-20T08:05:54.099811+0000","last_clean":"2021-02-20T08:05:54.099811+0000","last_became_active":"2021-02-20T08:05:54.098919+0000","last_became_peered":"2021-02-20T08:05:54.098919+0000","last_unstale":"2021-02-20T08:05:54.099811+0000","last_undegraded":"2021-02-20T08:05:54.099811+0000","last_fullsized":"2021-02-20T08:05:54.099811+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_clean_scrub_stamp":"2021-02-20T08:05:53.070215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:54.105952+0000","last_change":"2021-02-20T08:05:54.104099+0000","last_active":"2021-02-20T08:05:54.105952+0000","last_peered":"2021-02-20T08:05:54.105952+0000","last_clean":"2021-02-20T08:05:54.105952+0000","last_became_active":"2021-02-20T08:05:54.103760+0000","last_became_peered":"2021-02-20T08:05:54.103760+0000","last_unstale":"2021-02-20T08:05:54.105952+0000","last_undegraded":"2021-02-20T08:05:54.105952+0000","last_fullsized":"2021-02-20T08:05:54.105952+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_clean_scrub_stamp":"2021-02-20T08:05:53.070215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:54.104849+0000","last_change":"2021-02-20T08:05:54.101254+0000","last_active":"2021-02-20T08:05:54.104849+0000","last_peered":"2021-02-20T08:05:54.104849+0000","last_clean":"2021-02-20T08:05:54.104849+0000","last_became_active":"2021-02-20T08:05:54.100901+0000","last_became_peered":"2021-02-20T08:05:54.100901+0000","last_unstale":"2021-02-20T08:05:54.104849+0000","last_undegraded":"2021-02-20T08:05:54.104849+0000","last_fullsized":"2021-02-20T08:05:54.104849+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_clean_scrub_stamp":"2021-02-20T08:05:53.070215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:54.098700+0000","last_change":"2021-02-20T08:05:54.098700+0000","last_active":"2021-02-20T08:05:54.098700+0000","last_peered":"2021-02-20T08:05:54.098700+0000","last_clean":"2021-02-20T08:05:54.098700+0000","last_became_active":"2021-02-20T08:05:54.098321+0000","last_became_peered":"2021-02-20T08:05:54.098321+0000","last_unstale":"2021-02-20T08:05:54.098700+0000","last_undegraded":"2021-02-20T08:05:54.098700+0000","last_fullsized":"2021-02-20T08:05:54.098700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_clean_scrub_stamp":"2021-02-20T08:05:53.070215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:54.103377+0000","last_change":"2021-02-20T08:05:54.103377+0000","last_active":"2021-02-20T08:05:54.103377+0000","last_peered":"2021-02-20T08:05:54.103377+0000","last_clean":"2021-02-20T08:05:54.103377+0000","last_became_active":"2021-02-20T08:05:54.103020+0000","last_became_peered":"2021-02-20T08:05:54.103020+0000","last_unstale":"2021-02-20T08:05:54.103377+0000","last_undegraded":"2021-02-20T08:05:54.103377+0000","last_fullsized":"2021-02-20T08:05:54.103377+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_clean_scrub_stamp":"2021-02-20T08:05:53.070215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:54.105866+0000","last_change":"2021-02-20T08:05:54.102047+0000","last_active":"2021-02-20T08:05:54.105866+0000","last_peered":"2021-02-20T08:05:54.105866+0000","last_clean":"2021-02-20T08:05:54.105866+0000","last_became_active":"2021-02-20T08:05:54.101631+0000","last_became_peered":"2021-02-20T08:05:54.101631+0000","last_unstale":"2021-02-20T08:05:54.105866+0000","last_undegraded":"2021-02-20T08:05:54.105866+0000","last_fullsized":"2021-02-20T08:05:54.105866+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_clean_scrub_stamp":"2021-02-20T08:05:53.070215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4832,"kb_used_data":88,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94367008,"statfs":{"total":96636764160,"available":96631816192,"internally_reserved":0,"allocated":90112,"data_stored":22804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6879999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738370,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4896,"kb_used_data":88,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366944,"statfs":{"total":96636764160,"available":96631750656,"internally_reserved":0,"allocated":90112,"data_stored":22804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]}]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]}]}]}} 2021-02-20T08:05:59.137 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2021-02-20T08:05:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:59.432 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2021-02-20T08:05:59.443 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2021-02-20T08:05:58.000563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":14632,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":283100888,"statfs":{"total":289910292480,"available":289895309312,"internally_reserved":0,"allocated":278528,"data_stored":72292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:54.108806+0000","last_change":"2021-02-20T08:05:54.104973+0000","last_active":"2021-02-20T08:05:54.108806+0000","last_peered":"2021-02-20T08:05:54.108806+0000","last_clean":"2021-02-20T08:05:54.108806+0000","last_became_active":"2021-02-20T08:05:54.104618+0000","last_became_peered":"2021-02-20T08:05:54.104618+0000","last_unstale":"2021-02-20T08:05:54.108806+0000","last_undegraded":"2021-02-20T08:05:54.108806+0000","last_fullsized":"2021-02-20T08:05:54.108806+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_clean_scrub_stamp":"2021-02-20T08:05:53.070215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:54.108796+0000","last_change":"2021-02-20T08:05:54.104301+0000","last_active":"2021-02-20T08:05:54.108796+0000","last_peered":"2021-02-20T08:05:54.108796+0000","last_clean":"2021-02-20T08:05:54.108796+0000","last_became_active":"2021-02-20T08:05:54.103913+0000","last_became_peered":"2021-02-20T08:05:54.103913+0000","last_unstale":"2021-02-20T08:05:54.108796+0000","last_undegraded":"2021-02-20T08:05:54.108796+0000","last_fullsized":"2021-02-20T08:05:54.108796+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_clean_scrub_stamp":"2021-02-20T08:05:53.070215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:54.105899+0000","last_change":"2021-02-20T08:05:54.104028+0000","last_active":"2021-02-20T08:05:54.105899+0000","last_peered":"2021-02-20T08:05:54.105899+0000","last_clean":"2021-02-20T08:05:54.105899+0000","last_became_active":"2021-02-20T08:05:54.103735+0000","last_became_peered":"2021-02-20T08:05:54.103735+0000","last_unstale":"2021-02-20T08:05:54.105899+0000","last_undegraded":"2021-02-20T08:05:54.105899+0000","last_fullsized":"2021-02-20T08:05:54.105899+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_clean_scrub_stamp":"2021-02-20T08:05:53.070215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:54.099811+0000","last_change":"2021-02-20T08:05:54.099811+0000","last_active":"2021-02-20T08:05:54.099811+0000","last_peered":"2021-02-20T08:05:54.099811+0000","last_clean":"2021-02-20T08:05:54.099811+0000","last_became_active":"2021-02-20T08:05:54.098919+0000","last_became_peered":"2021-02-20T08:05:54.098919+0000","last_unstale":"2021-02-20T08:05:54.099811+0000","last_undegraded":"2021-02-20T08:05:54.099811+0000","last_fullsized":"2021-02-20T08:05:54.099811+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_clean_scrub_stamp":"2021-02-20T08:05:53.070215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:54.105952+0000","last_change":"2021-02-20T08:05:54.104099+0000","last_active":"2021-02-20T08:05:54.105952+0000","last_peered":"2021-02-20T08:05:54.105952+0000","last_clean":"2021-02-20T08:05:54.105952+0000","last_became_active":"2021-02-20T08:05:54.103760+0000","last_became_peered":"2021-02-20T08:05:54.103760+0000","last_unstale":"2021-02-20T08:05:54.105952+0000","last_undegraded":"2021-02-20T08:05:54.105952+0000","last_fullsized":"2021-02-20T08:05:54.105952+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_clean_scrub_stamp":"2021-02-20T08:05:53.070215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:54.104849+0000","last_change":"2021-02-20T08:05:54.101254+0000","last_active":"2021-02-20T08:05:54.104849+0000","last_peered":"2021-02-20T08:05:54.104849+0000","last_clean":"2021-02-20T08:05:54.104849+0000","last_became_active":"2021-02-20T08:05:54.100901+0000","last_became_peered":"2021-02-20T08:05:54.100901+0000","last_unstale":"2021-02-20T08:05:54.104849+0000","last_undegraded":"2021-02-20T08:05:54.104849+0000","last_fullsized":"2021-02-20T08:05:54.104849+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_clean_scrub_stamp":"2021-02-20T08:05:53.070215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:54.098700+0000","last_change":"2021-02-20T08:05:54.098700+0000","last_active":"2021-02-20T08:05:54.098700+0000","last_peered":"2021-02-20T08:05:54.098700+0000","last_clean":"2021-02-20T08:05:54.098700+0000","last_became_active":"2021-02-20T08:05:54.098321+0000","last_became_peered":"2021-02-20T08:05:54.098321+0000","last_unstale":"2021-02-20T08:05:54.098700+0000","last_undegraded":"2021-02-20T08:05:54.098700+0000","last_fullsized":"2021-02-20T08:05:54.098700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_clean_scrub_stamp":"2021-02-20T08:05:53.070215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:54.103377+0000","last_change":"2021-02-20T08:05:54.103377+0000","last_active":"2021-02-20T08:05:54.103377+0000","last_peered":"2021-02-20T08:05:54.103377+0000","last_clean":"2021-02-20T08:05:54.103377+0000","last_became_active":"2021-02-20T08:05:54.103020+0000","last_became_peered":"2021-02-20T08:05:54.103020+0000","last_unstale":"2021-02-20T08:05:54.103377+0000","last_undegraded":"2021-02-20T08:05:54.103377+0000","last_fullsized":"2021-02-20T08:05:54.103377+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_clean_scrub_stamp":"2021-02-20T08:05:53.070215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T08:05:54.105866+0000","last_change":"2021-02-20T08:05:54.102047+0000","last_active":"2021-02-20T08:05:54.105866+0000","last_peered":"2021-02-20T08:05:54.105866+0000","last_clean":"2021-02-20T08:05:54.105866+0000","last_became_active":"2021-02-20T08:05:54.101631+0000","last_became_peered":"2021-02-20T08:05:54.101631+0000","last_unstale":"2021-02-20T08:05:54.105866+0000","last_undegraded":"2021-02-20T08:05:54.105866+0000","last_fullsized":"2021-02-20T08:05:54.105866+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T08:05:53.070215+0000","last_clean_scrub_stamp":"2021-02-20T08:05:53.070215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4832,"kb_used_data":88,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94367008,"statfs":{"total":96636764160,"available":96631816192,"internally_reserved":0,"allocated":90112,"data_stored":22804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6879999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738370,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4896,"kb_used_data":88,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366944,"statfs":{"total":96636764160,"available":96631750656,"internally_reserved":0,"allocated":90112,"data_stored":22804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]}]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]}]}]}} 2021-02-20T08:05:59.444 INFO:tasks.ceph.ceph_manager.cluster2:clean! 2021-02-20T08:05:59.445 INFO:tasks.ceph:Waiting until ceph cluster cluster2 is healthy... 2021-02-20T08:05:59.445 INFO:tasks.ceph.ceph_manager.cluster2:wait_until_healthy 2021-02-20T08:05:59.446 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 health --format=json 2021-02-20T08:05:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: 2021-02-20T08:05:59.793 INFO:teuthology.orchestra.run.smithi090.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2021-02-20T08:05:59.806 INFO:tasks.ceph.ceph_manager.cluster2:wait_until_healthy done 2021-02-20T08:05:59.807 INFO:teuthology.run_tasks:Running task exec... 2021-02-20T08:05:59.825 INFO:teuthology.task.exec:Executing custom commands... 2021-02-20T08:05:59.825 INFO:teuthology.task.exec:Running commands on role cluster1.client.mirror.0 host ubuntu@smithi090.front.sepia.ceph.com 2021-02-20T08:05:59.826 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror mon '"'"'profile rbd-mirror-peer'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T08:06:00.206 INFO:teuthology.orchestra.run.smithi090.stderr:updated caps for client.mirror 2021-02-20T08:06:00.221 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.0 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T08:06:00.597 INFO:teuthology.orchestra.run.smithi090.stderr:updated caps for client.mirror.0 2021-02-20T08:06:00.610 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.1 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T08:06:00.976 INFO:teuthology.orchestra.run.smithi090.stderr:updated caps for client.mirror.1 2021-02-20T08:06:00.991 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.2 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T08:06:01.364 INFO:teuthology.orchestra.run.smithi090.stderr:updated caps for client.mirror.2 2021-02-20T08:06:01.377 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.3 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T08:06:01.750 INFO:teuthology.orchestra.run.smithi090.stderr:updated caps for client.mirror.3 2021-02-20T08:06:01.764 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror mon '"'"'profile rbd-mirror-peer'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T08:06:02.141 INFO:teuthology.orchestra.run.smithi090.stderr:updated caps for client.mirror 2021-02-20T08:06:02.154 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.0 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T08:06:02.522 INFO:teuthology.orchestra.run.smithi090.stderr:updated caps for client.mirror.0 2021-02-20T08:06:02.536 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.1 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T08:06:02.911 INFO:teuthology.orchestra.run.smithi090.stderr:updated caps for client.mirror.1 2021-02-20T08:06:02.925 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.2 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T08:06:03.302 INFO:teuthology.orchestra.run.smithi090.stderr:updated caps for client.mirror.2 2021-02-20T08:06:03.317 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.3 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T08:06:03.693 INFO:teuthology.orchestra.run.smithi090.stderr:updated caps for client.mirror.3 2021-02-20T08:06:03.706 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T08:06:03.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T08:06:03.766 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T08:06:03.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T08:06:03.769 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T08:06:03.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T08:06:03.780 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T08:06:03.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T08:06:03.782 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T08:06:03.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T08:06:03.798 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T08:06:03.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T08:06:03.801 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T08:06:03.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T08:06:03.812 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T08:06:03.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T08:06:03.815 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T08:06:03.831 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T08:06:03.831 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T08:06:03.834 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T08:06:03.834 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T08:06:03.850 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T08:06:03.850 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T08:06:03.852 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T08:06:03.853 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T08:06:03.908 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T08:06:03.909 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T08:06:03.911 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T08:06:03.911 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T08:06:03.926 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T08:06:03.927 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T08:06:03.929 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T08:06:03.929 INFO:teuthology.run_tasks:Running task rbd-mirror-thrash... 2021-02-20T08:06:03.946 INFO:tasks.rbd_mirror_thrash:rbd_mirror_thrash using random seed: 1613808363 2021-02-20T08:06:03.947 DEBUG:tasks.rbd_mirror_thrash:Yielding 2021-02-20T08:06:03.947 INFO:teuthology.run_tasks:Running task rbd-mirror-thrash... 2021-02-20T08:06:03.960 INFO:tasks.rbd_mirror_thrash:rbd_mirror_thrash using random seed: 1613808363 2021-02-20T08:06:03.960 DEBUG:tasks.rbd_mirror_thrash:Yielding 2021-02-20T08:06:03.961 INFO:teuthology.run_tasks:Running task workunit... 2021-02-20T08:06:04.006 INFO:tasks.workunit:Pulling workunits from ref d318d42d502ea13c78f2af2499efd19dcc7e174b 2021-02-20T08:06:04.007 INFO:tasks.workunit:Making a separate scratch dir for every client... 2021-02-20T08:06:04.008 DEBUG:teuthology.orchestra.run.smithi090:> stat -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2021-02-20T08:06:04.009 INFO:tasks.rbd_mirror_thrash:starting thrash for cluster cluster1 2021-02-20T08:06:04.009 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T08:06:04.009 INFO:tasks.rbd_mirror_thrash:starting thrash for cluster cluster2 2021-02-20T08:06:04.009 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T08:06:04.015 INFO:teuthology.orchestra.run.smithi090.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.cluster1.mirror': No such file or directory 2021-02-20T08:06:04.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:06:04.016 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2021-02-20T08:06:04.063 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.cluster1.mirror 2021-02-20T08:06:04.064 DEBUG:teuthology.orchestra.run.smithi090:> cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror && mkdir -- client.mirror 2021-02-20T08:06:04.112 INFO:tasks.workunit:timeout=6h 2021-02-20T08:06:04.112 INFO:tasks.workunit:cleanup=True 2021-02-20T08:06:04.113 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf /home/ubuntu/cephtest/clone.cluster1.client.mirror && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.cluster1.client.mirror && cd /home/ubuntu/cephtest/clone.cluster1.client.mirror && git checkout d318d42d502ea13c78f2af2499efd19dcc7e174b 2021-02-20T08:06:04.161 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:Cloning into '/home/ubuntu/cephtest/clone.cluster1.client.mirror'... 2021-02-20T08:06:38.635 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:Note: checking out 'd318d42d502ea13c78f2af2499efd19dcc7e174b'. 2021-02-20T08:06:38.636 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:06:38.636 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2021-02-20T08:06:38.637 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:changes and commit them, and you can discard any commits you make in this 2021-02-20T08:06:38.637 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:state without impacting any branches by performing another checkout. 2021-02-20T08:06:38.637 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:06:38.637 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:If you want to create a new branch to retain commits you create, you may 2021-02-20T08:06:38.638 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:do so (now or later) by using -b with the checkout command again. Example: 2021-02-20T08:06:38.638 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:06:38.638 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: git checkout -b 2021-02-20T08:06:38.638 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:06:38.638 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:HEAD is now at d318d42d50 rbd-mirror: delay update snapshot mirror image state by dillaman · Pull Request #39432 · ceph/ceph · GitHub 2021-02-20T08:06:38.640 DEBUG:teuthology.orchestra.run.smithi090:> cd -- /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.cluster1.client.mirror 2021-02-20T08:06:38.687 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2021-02-20T08:06:38.691 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/direct_io' 2021-02-20T08:06:38.691 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2021-02-20T08:06:39.522 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2021-02-20T08:06:39.678 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2021-02-20T08:06:39.730 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/direct_io' 2021-02-20T08:06:39.732 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/fs' 2021-02-20T08:06:39.732 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2021-02-20T08:06:39.791 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/fs' 2021-02-20T08:06:39.798 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2021-02-20T08:06:39.798 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/home/ubuntu/cephtest/workunits.list.cluster1.client.mirror of=/dev/stdout 2021-02-20T08:06:39.847 INFO:tasks.workunit:Running workunits matching rbd/rbd_mirror_stress.sh on cluster1.client.mirror... 2021-02-20T08:06:39.848 INFO:tasks.workunit:Running workunit rbd/rbd_mirror_stress.sh... 2021-02-20T08:06:39.848 DEBUG:teuthology.orchestra.run.smithi090:workunit test rbd/rbd_mirror_stress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d318d42d502ea13c78f2af2499efd19dcc7e174b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS='' MIRROR_IMAGE_MODE=snapshot MIRROR_POOL_MODE=image RBD_IMAGE_FEATURES=layering,exclusive-lock,object-map,fast-diff RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 RBD_MIRROR_USE_RBD_MIRROR=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh 2021-02-20T08:06:39.905 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ IMAGE_COUNT=50 2021-02-20T08:06:39.905 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ export LOCKDEP=0 2021-02-20T08:06:39.905 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ dirname /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh 2021-02-20T08:06:39.907 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ . /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_helpers.sh 2021-02-20T08:06:39.907 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ type xmlstarlet 2021-02-20T08:06:39.907 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ XMLSTARLET=xmlstarlet 2021-02-20T08:06:39.908 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ RBD_MIRROR_INSTANCES=4 2021-02-20T08:06:39.908 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CLUSTER1=cluster1 2021-02-20T08:06:39.908 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CLUSTER2=cluster2 2021-02-20T08:06:39.909 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ PEER_CLUSTER_SUFFIX= 2021-02-20T08:06:39.909 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ POOL=mirror 2021-02-20T08:06:39.910 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ PARENT_POOL=mirror_parent 2021-02-20T08:06:39.910 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ NS1=ns1 2021-02-20T08:06:39.910 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ NS2=ns2 2021-02-20T08:06:39.910 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ TEMPDIR= 2021-02-20T08:06:39.911 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_ID=mirror 2021-02-20T08:06:39.911 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ RBD_IMAGE_FEATURES=layering,exclusive-lock,object-map,fast-diff 2021-02-20T08:06:39.911 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ MIRROR_USER_ID_PREFIX=mirror. 2021-02-20T08:06:39.911 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ MIRROR_POOL_MODE=image 2021-02-20T08:06:39.912 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ MIRROR_IMAGE_MODE=snapshot 2021-02-20T08:06:39.912 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ export CEPH_ARGS=--id mirror 2021-02-20T08:06:39.912 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ LAST_MIRROR_INSTANCE=3 2021-02-20T08:06:39.912 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ dirname /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh 2021-02-20T08:06:39.914 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ readlink -f /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/../../../src 2021-02-20T08:06:39.914 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror/src 2021-02-20T08:06:39.915 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_BIN=. 2021-02-20T08:06:39.916 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_SRC=. 2021-02-20T08:06:39.916 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -e CMakeCache.txt ] 2021-02-20T08:06:39.916 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 -gt 0 ] 2021-02-20T08:06:39.917 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ setup 2021-02-20T08:06:39.917 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local c 2021-02-20T08:06:39.917 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ trap cleanup $? INT TERM EXIT 2021-02-20T08:06:39.917 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ setup_tempdir 2021-02-20T08:06:39.917 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:06:39.918 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mktemp -d 2021-02-20T08:06:39.918 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ TEMPDIR=/tmp/tmp.1RV4oXZGap 2021-02-20T08:06:39.918 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -z 1 ] 2021-02-20T08:06:39.918 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ setup_pools cluster1 cluster2 2021-02-20T08:06:39.918 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:06:39.919 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local remote_cluster=cluster2 2021-02-20T08:06:39.919 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mon_map_file 2021-02-20T08:06:39.919 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mon_addr 2021-02-20T08:06:39.919 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local admin_key_file 2021-02-20T08:06:39.920 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local uuid 2021-02-20T08:06:39.920 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_ARGS= ceph --cluster cluster1 osd pool create mirror 64 64 2021-02-20T08:06:40.859 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:pool 'mirror' already exists 2021-02-20T08:06:40.869 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_ARGS= ceph --cluster cluster1 osd pool create mirror_parent 64 64 2021-02-20T08:06:41.873 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:pool 'mirror_parent' already exists 2021-02-20T08:06:41.882 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_ARGS= rbd --cluster cluster1 pool init mirror 2021-02-20T08:06:44.803 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_ARGS= rbd --cluster cluster1 pool init mirror_parent 2021-02-20T08:06:47.825 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:06:47.826 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror pool enable --site-name cluster1 mirror image 2021-02-20T08:06:47.888 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 mirror pool enable mirror_parent image 2021-02-20T08:06:47.942 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 namespace create mirror/ns1 2021-02-20T08:06:47.991 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 namespace create mirror/ns2 2021-02-20T08:06:48.039 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 mirror pool enable mirror/ns1 image 2021-02-20T08:06:48.094 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 mirror pool enable mirror/ns2 image 2021-02-20T08:06:48.151 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -z ] 2021-02-20T08:06:48.151 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -z ] 2021-02-20T08:06:48.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ peer_add cluster1 mirror cluster2 2021-02-20T08:06:48.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:06:48.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:48.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:06:48.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:48.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local client_cluster=cluster2 2021-02-20T08:06:48.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:48.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local remote_cluster=cluster2 2021-02-20T08:06:48.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local uuid_var_name 2021-02-20T08:06:48.154 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:06:48.154 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local error_code 2021-02-20T08:06:48.154 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local peer_uuid 2021-02-20T08:06:48.154 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:06:48.155 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 mirror pool peer add mirror cluster2 2021-02-20T08:06:48.202 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ peer_uuid=233e14bb-fd37-4420-b56a-db9237ba9194 2021-02-20T08:06:48.203 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=0 2021-02-20T08:06:48.203 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:06:48.203 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 -eq 17 ] 2021-02-20T08:06:48.203 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 0 -eq 0 2021-02-20T08:06:48.203 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:06:48.204 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:06:48.204 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ peer_add cluster1 mirror_parent cluster2 2021-02-20T08:06:48.204 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:06:48.204 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:48.205 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror_parent 2021-02-20T08:06:48.205 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:48.205 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local client_cluster=cluster2 2021-02-20T08:06:48.205 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:48.205 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local remote_cluster=cluster2 2021-02-20T08:06:48.206 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local uuid_var_name 2021-02-20T08:06:48.206 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:06:48.206 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local error_code 2021-02-20T08:06:48.206 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local peer_uuid 2021-02-20T08:06:48.207 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:06:48.207 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 mirror pool peer add mirror_parent cluster2 2021-02-20T08:06:48.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ peer_uuid=a897d176-90f0-4acc-a699-a277b78d9c0a 2021-02-20T08:06:48.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=0 2021-02-20T08:06:48.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:06:48.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 -eq 17 ] 2021-02-20T08:06:48.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 0 -eq 0 2021-02-20T08:06:48.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:06:48.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:06:48.256 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ setup_pools cluster2 cluster1 2021-02-20T08:06:48.256 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:06:48.256 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local remote_cluster=cluster1 2021-02-20T08:06:48.256 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mon_map_file 2021-02-20T08:06:48.256 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mon_addr 2021-02-20T08:06:48.257 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local admin_key_file 2021-02-20T08:06:48.257 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local uuid 2021-02-20T08:06:48.257 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_ARGS= ceph --cluster cluster2 osd pool create mirror 64 64 2021-02-20T08:06:48.950 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:pool 'mirror' already exists 2021-02-20T08:06:48.960 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_ARGS= ceph --cluster cluster2 osd pool create mirror_parent 64 64 2021-02-20T08:06:49.985 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:pool 'mirror_parent' already exists 2021-02-20T08:06:49.996 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_ARGS= rbd --cluster cluster2 pool init mirror 2021-02-20T08:06:52.878 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_ARGS= rbd --cluster cluster2 pool init mirror_parent 2021-02-20T08:06:55.906 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:06:55.906 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror pool enable --site-name cluster2 mirror image 2021-02-20T08:06:55.964 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 mirror pool enable mirror_parent image 2021-02-20T08:06:56.002 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 namespace create mirror/ns1 2021-02-20T08:06:56.046 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 namespace create mirror/ns2 2021-02-20T08:06:56.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 mirror pool enable mirror/ns1 image 2021-02-20T08:06:56.138 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 mirror pool enable mirror/ns2 image 2021-02-20T08:06:56.188 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -z ] 2021-02-20T08:06:56.189 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -z ] 2021-02-20T08:06:56.189 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ peer_add cluster2 mirror cluster1 2021-02-20T08:06:56.189 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:06:56.190 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:56.190 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:06:56.190 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:56.190 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local client_cluster=cluster1 2021-02-20T08:06:56.190 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:56.191 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local remote_cluster=cluster1 2021-02-20T08:06:56.191 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local uuid_var_name 2021-02-20T08:06:56.191 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:06:56.191 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local error_code 2021-02-20T08:06:56.191 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local peer_uuid 2021-02-20T08:06:56.192 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:06:56.192 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 mirror pool peer add mirror cluster1 2021-02-20T08:06:56.237 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ peer_uuid=36088fa1-4922-4122-9133-0b47fbf3162e 2021-02-20T08:06:56.237 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=0 2021-02-20T08:06:56.238 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:06:56.238 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 -eq 17 ] 2021-02-20T08:06:56.238 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 0 -eq 0 2021-02-20T08:06:56.238 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:06:56.238 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:06:56.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ peer_add cluster2 mirror_parent cluster1 2021-02-20T08:06:56.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:06:56.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:56.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror_parent 2021-02-20T08:06:56.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:56.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local client_cluster=cluster1 2021-02-20T08:06:56.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:56.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local remote_cluster=cluster1 2021-02-20T08:06:56.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local uuid_var_name 2021-02-20T08:06:56.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:06:56.241 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local error_code 2021-02-20T08:06:56.241 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local peer_uuid 2021-02-20T08:06:56.241 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:06:56.241 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 mirror pool peer add mirror_parent cluster1 2021-02-20T08:06:56.282 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ peer_uuid=c32efbe9-ab54-4cf6-9175-0b4497f0bb00 2021-02-20T08:06:56.283 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=0 2021-02-20T08:06:56.283 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:06:56.283 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 -eq 17 ] 2021-02-20T08:06:56.284 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 0 -eq 0 2021-02-20T08:06:56.284 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:06:56.284 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:06:56.284 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:06:56.284 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ start_mirrors cluster1 2021-02-20T08:06:56.285 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:06:56.285 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ seq 0 3 2021-02-20T08:06:56.302 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ start_mirror cluster1:0 2021-02-20T08:06:56.302 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:0 2021-02-20T08:06:56.302 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:56.302 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:06:56.303 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:06:56.303 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:56.303 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:56.303 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:56.304 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:06:56.304 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:06:56.304 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:06:56.304 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:56.304 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:06:56.305 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:06:56.305 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -n 1 2021-02-20T08:06:56.305 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 2021-02-20T08:06:56.305 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ start_mirror cluster1:1 2021-02-20T08:06:56.306 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:1 2021-02-20T08:06:56.306 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:56.306 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T08:06:56.306 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:1 2021-02-20T08:06:56.307 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:56.307 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:56.307 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:56.307 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:06:56.308 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:06:56.308 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:06:56.308 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:56.308 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=1 2021-02-20T08:06:56.308 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:06:56.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -n 1 2021-02-20T08:06:56.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 2021-02-20T08:06:56.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ start_mirror cluster1:2 2021-02-20T08:06:56.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:2 2021-02-20T08:06:56.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:56.310 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T08:06:56.310 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:2 2021-02-20T08:06:56.310 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:56.310 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:56.311 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:56.311 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:06:56.311 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:06:56.311 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:06:56.311 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:56.312 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=2 2021-02-20T08:06:56.312 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:06:56.312 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -n 1 2021-02-20T08:06:56.312 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 2021-02-20T08:06:56.312 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ start_mirror cluster1:3 2021-02-20T08:06:56.313 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:3 2021-02-20T08:06:56.313 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:56.313 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T08:06:56.313 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:3 2021-02-20T08:06:56.314 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:56.314 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:56.314 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:56.314 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=3 2021-02-20T08:06:56.314 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T08:06:56.315 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:06:56.315 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:56.315 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=3 2021-02-20T08:06:56.315 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=3 2021-02-20T08:06:56.315 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -n 1 2021-02-20T08:06:56.316 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 2021-02-20T08:06:56.316 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ start_mirrors cluster2 2021-02-20T08:06:56.316 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:06:56.316 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ seq 0 3 2021-02-20T08:06:56.316 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ start_mirror cluster2:0 2021-02-20T08:06:56.317 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2:0 2021-02-20T08:06:56.317 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:56.317 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster2:0 cluster instance 2021-02-20T08:06:56.317 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster2:0 2021-02-20T08:06:56.317 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:56.318 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:56.318 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster2 2021-02-20T08:06:56.318 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:06:56.319 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster2:0 ] 2021-02-20T08:06:56.319 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster2 2021-02-20T08:06:56.319 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster2 2021-02-20T08:06:56.320 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:06:56.320 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:06:56.321 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -n 1 2021-02-20T08:06:56.321 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 2021-02-20T08:06:56.321 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ start_mirror cluster2:1 2021-02-20T08:06:56.322 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2:1 2021-02-20T08:06:56.322 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:56.322 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster2:1 cluster instance 2021-02-20T08:06:56.323 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster2:1 2021-02-20T08:06:56.323 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:56.323 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:56.324 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster2 2021-02-20T08:06:56.324 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:06:56.324 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 1 = cluster2:1 ] 2021-02-20T08:06:56.325 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster2 2021-02-20T08:06:56.325 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster2 2021-02-20T08:06:56.326 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=1 2021-02-20T08:06:56.326 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:06:56.326 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -n 1 2021-02-20T08:06:56.327 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 2021-02-20T08:06:56.327 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ start_mirror cluster2:2 2021-02-20T08:06:56.327 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2:2 2021-02-20T08:06:56.328 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:56.328 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster2:2 cluster instance 2021-02-20T08:06:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster2:2 2021-02-20T08:06:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:56.330 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster2 2021-02-20T08:06:56.330 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:06:56.330 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 2 = cluster2:2 ] 2021-02-20T08:06:56.331 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster2 2021-02-20T08:06:56.331 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster2 2021-02-20T08:06:56.331 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=2 2021-02-20T08:06:56.332 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:06:56.332 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -n 1 2021-02-20T08:06:56.332 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 2021-02-20T08:06:56.333 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ start_mirror cluster2:3 2021-02-20T08:06:56.333 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2:3 2021-02-20T08:06:56.334 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:56.334 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster2:3 cluster instance 2021-02-20T08:06:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster2:3 2021-02-20T08:06:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster2 2021-02-20T08:06:56.336 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=3 2021-02-20T08:06:56.336 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 3 = cluster2:3 ] 2021-02-20T08:06:56.336 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster2 2021-02-20T08:06:56.336 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster2 2021-02-20T08:06:56.336 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=3 2021-02-20T08:06:56.337 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=3 2021-02-20T08:06:56.337 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -n 1 2021-02-20T08:06:56.337 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 2021-02-20T08:06:56.337 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ testlog TEST: add image and test replay after client crashes 2021-02-20T08:06:56.338 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ tee -a /tmp/tmp.1RV4oXZGap/rbd-mirror.test.log 2021-02-20T08:06:56.338 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ date +%F %T 2021-02-20T08:06:56.338 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo 2021-02-20 08:06:56 TEST: add image and test replay after client crashes 2021-02-20T08:06:56.338 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:2021-02-20 08:06:56 TEST: add image and test replay after client crashes 2021-02-20T08:06:56.338 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=test 2021-02-20T08:06:56.339 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror test snapshot 512M 2021-02-20T08:06:56.339 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:06:56.339 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:56.339 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:06:56.339 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:56.340 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:06:56.340 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:56.340 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:06:56.340 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:06:56.341 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:56.341 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror test 512M 2021-02-20T08:06:56.341 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:06:56.341 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:56.341 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:06:56.342 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:56.342 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:06:56.342 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:56.342 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:06:56.342 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 512M ] 2021-02-20T08:06:56.343 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=512M 2021-02-20T08:06:56.343 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:56.343 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/test 2021-02-20T08:06:56.370 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:06:56.371 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror test snapshot 2021-02-20T08:06:56.371 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:06:56.371 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:06:56.371 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:06:56.372 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:06:56.372 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/test snapshot 2021-02-20T08:06:56.986 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:06:56.998 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T08:06:56.998 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:06:56.999 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:06:56.999 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:06:56.999 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T08:06:56.999 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:06:57.000 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:06:57.000 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:06:57.000 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state=started 2021-02-20T08:06:57.000 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:06:57.000 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:06:57.999 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T08:06:58.000 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:06:58.000 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:06:58.000 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:06:58.000 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local test_state=started 2021-02-20T08:06:58.000 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_result 2021-02-20T08:06:58.001 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_state=stopped 2021-02-20T08:06:58.002 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T08:06:58.002 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_instance=cluster1 2021-02-20T08:06:58.003 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ + shift 2021-02-20T08:06:58.003 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:grep -i state 2021-02-20T08:06:58.003 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:06:58.004 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance=cluster1 2021-02-20T08:06:58.004 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loop_instance 2021-02-20T08:06:58.004 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0 2021-02-20T08:06:58.004 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:06:58.005 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ seq 0 3 2021-02-20T08:06:58.006 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:06:58.007 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:0 2021-02-20T08:06:58.007 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:58.007 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:58.008 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:06:58.008 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:06:58.008 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:58.008 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:58.008 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:58.009 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:06:58.009 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:06:58.009 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:06:58.009 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:58.009 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:06:58.010 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:06:58.010 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:06:58.010 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:06:58.010 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:06:58.011 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:58.011 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:06:58.011 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:06:58.011 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:06:58.011 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:58.012 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:58.012 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:06:58.012 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:06:58.012 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:06:58.012 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:06:58.013 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:06:58.013 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:06:58.013 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:06:58.026 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:06:58.026 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:06:58.026 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:06:58.027 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:06:58.130 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:06:58.131 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:06:58.131 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:admin_socket: invalid command 2021-02-20T08:06:58.133 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T08:06:58.133 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:1 2021-02-20T08:06:58.133 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:58.133 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:58.133 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T08:06:58.134 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:1 2021-02-20T08:06:58.134 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:58.134 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:58.134 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:58.135 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:06:58.135 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:06:58.135 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:06:58.135 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:58.135 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=1 2021-02-20T08:06:58.136 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:06:58.136 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T08:06:58.136 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:1 2021-02-20T08:06:58.136 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:06:58.136 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:58.137 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T08:06:58.137 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:1 2021-02-20T08:06:58.137 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:06:58.137 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:58.137 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:58.138 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:06:58.138 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:06:58.139 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:06:58.139 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:06:58.139 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=1 2021-02-20T08:06:58.140 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:06:58.140 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T08:06:58.151 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:06:58.151 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:06:58.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:06:58.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T08:06:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:06:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:06:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:admin_socket: invalid command 2021-02-20T08:06:58.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T08:06:58.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:2 2021-02-20T08:06:58.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:58.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:58.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T08:06:58.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:2 2021-02-20T08:06:58.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:58.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:58.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:58.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:06:58.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:06:58.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:06:58.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:58.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=2 2021-02-20T08:06:58.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:06:58.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T08:06:58.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:2 2021-02-20T08:06:58.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:06:58.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:58.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T08:06:58.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:2 2021-02-20T08:06:58.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:06:58.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:58.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:58.256 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:06:58.256 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:06:58.256 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:06:58.256 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:06:58.256 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=2 2021-02-20T08:06:58.257 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:06:58.257 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T08:06:58.271 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:06:58.271 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:06:58.271 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:06:58.271 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T08:06:58.366 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:06:58.366 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:06:58.366 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:admin_socket: invalid command 2021-02-20T08:06:58.368 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T08:06:58.369 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:3 2021-02-20T08:06:58.369 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:58.369 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:58.369 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T08:06:58.370 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:3 2021-02-20T08:06:58.370 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:58.370 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:58.370 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:58.370 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=3 2021-02-20T08:06:58.371 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T08:06:58.371 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:06:58.371 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:58.371 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=3 2021-02-20T08:06:58.371 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=3 2021-02-20T08:06:58.372 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T08:06:58.372 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:3 2021-02-20T08:06:58.372 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:06:58.372 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:58.372 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T08:06:58.373 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:3 2021-02-20T08:06:58.373 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:06:58.373 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:58.373 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:58.373 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=3 2021-02-20T08:06:58.374 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T08:06:58.374 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:06:58.374 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:06:58.374 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=3 2021-02-20T08:06:58.374 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=3 2021-02-20T08:06:58.375 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T08:06:58.389 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:06:58.389 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:06:58.389 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:06:58.390 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T08:06:58.478 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:06:58.478 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:06:58.478 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:admin_socket: invalid command 2021-02-20T08:06:58.479 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:06:59.480 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:06:59.480 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ seq 0 3 2021-02-20T08:06:59.483 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:06:59.483 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:0 2021-02-20T08:06:59.484 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:59.484 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:59.484 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:06:59.485 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:06:59.485 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:59.486 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:59.486 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:59.487 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:06:59.487 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:06:59.487 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:06:59.488 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:59.488 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:06:59.488 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:06:59.489 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:06:59.489 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:06:59.489 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:06:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:06:59.496 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:06:59.496 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:06:59.496 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:59.497 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:59.497 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:06:59.497 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:06:59.497 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:06:59.498 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:06:59.498 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:06:59.498 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:06:59.498 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:06:59.505 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:06:59.505 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:06:59.506 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:06:59.506 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:06:59.608 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:06:59.609 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:06:59.609 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:admin_socket: invalid command 2021-02-20T08:06:59.610 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T08:06:59.611 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:1 2021-02-20T08:06:59.611 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:59.611 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:59.611 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T08:06:59.612 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:1 2021-02-20T08:06:59.612 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:59.612 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:59.612 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:59.613 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:06:59.613 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:06:59.613 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:06:59.613 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:59.613 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=1 2021-02-20T08:06:59.614 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:06:59.614 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T08:06:59.614 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:1 2021-02-20T08:06:59.614 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:06:59.615 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:59.615 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T08:06:59.616 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:1 2021-02-20T08:06:59.616 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:06:59.617 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:59.617 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:59.617 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:06:59.617 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:06:59.617 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:06:59.618 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:06:59.618 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=1 2021-02-20T08:06:59.618 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:06:59.618 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T08:06:59.632 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:06:59.632 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:06:59.633 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:06:59.633 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T08:06:59.720 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:06:59.721 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:06:59.721 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:admin_socket: invalid command 2021-02-20T08:06:59.722 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T08:06:59.722 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:2 2021-02-20T08:06:59.723 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:59.723 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:59.723 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T08:06:59.723 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:2 2021-02-20T08:06:59.723 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:59.724 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:59.724 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:59.724 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:06:59.724 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:06:59.724 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:06:59.725 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:59.725 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=2 2021-02-20T08:06:59.725 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:06:59.725 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T08:06:59.725 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:2 2021-02-20T08:06:59.726 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:06:59.726 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:59.726 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T08:06:59.726 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:2 2021-02-20T08:06:59.726 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:06:59.727 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:59.727 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:59.727 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:06:59.727 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:06:59.728 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:06:59.728 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:06:59.728 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=2 2021-02-20T08:06:59.728 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:06:59.728 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T08:06:59.742 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:06:59.742 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:06:59.742 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:06:59.743 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T08:06:59.836 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:06:59.836 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:06:59.836 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:admin_socket: invalid command 2021-02-20T08:06:59.838 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T08:06:59.838 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:3 2021-02-20T08:06:59.839 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:06:59.839 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:59.839 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T08:06:59.839 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:3 2021-02-20T08:06:59.839 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:59.840 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:59.840 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:59.840 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=3 2021-02-20T08:06:59.840 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T08:06:59.840 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:06:59.841 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:59.841 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=3 2021-02-20T08:06:59.841 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=3 2021-02-20T08:06:59.841 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T08:06:59.841 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:3 2021-02-20T08:06:59.842 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:06:59.842 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:06:59.842 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T08:06:59.842 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:3 2021-02-20T08:06:59.843 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:06:59.843 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:06:59.843 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:06:59.843 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=3 2021-02-20T08:06:59.843 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T08:06:59.844 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:06:59.844 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:06:59.844 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=3 2021-02-20T08:06:59.845 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=3 2021-02-20T08:06:59.845 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T08:06:59.860 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:06:59.861 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:06:59.861 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:06:59.861 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T08:06:59.948 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:06:59.949 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:06:59.949 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:admin_socket: invalid command 2021-02-20T08:06:59.950 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 2 2021-02-20T08:07:01.951 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:07:01.951 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ seq 0 3 2021-02-20T08:07:01.953 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:07:01.954 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:0 2021-02-20T08:07:01.954 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:07:01.954 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:07:01.954 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:07:01.955 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:07:01.955 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:07:01.955 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:07:01.955 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:07:01.955 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:07:01.956 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:07:01.956 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:07:01.956 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:07:01.957 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:07:01.957 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:07:01.957 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:07:01.957 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:07:01.957 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:07:01.958 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:07:01.958 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:07:01.958 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:07:01.959 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:07:01.959 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:07:01.959 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:07:01.960 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:07:01.960 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:07:01.960 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:07:01.960 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:07:01.961 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:07:01.961 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:07:01.961 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:07:01.977 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:07:01.978 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:07:01.978 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:07:01.978 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:07:02.081 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:07:02.081 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:07:02.082 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:admin_socket: invalid command 2021-02-20T08:07:02.086 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T08:07:02.087 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:1 2021-02-20T08:07:02.087 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:07:02.088 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:07:02.088 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T08:07:02.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:1 2021-02-20T08:07:02.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:07:02.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:07:02.090 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:07:02.090 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:07:02.091 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:07:02.091 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:07:02.091 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:07:02.092 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=1 2021-02-20T08:07:02.092 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:07:02.092 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T08:07:02.092 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:1 2021-02-20T08:07:02.093 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:07:02.093 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:07:02.093 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T08:07:02.093 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:1 2021-02-20T08:07:02.094 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:07:02.094 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:07:02.095 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:07:02.095 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:07:02.095 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:07:02.095 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:07:02.095 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:07:02.096 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=1 2021-02-20T08:07:02.096 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:07:02.096 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T08:07:02.102 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:07:02.102 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:07:02.103 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:07:02.103 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T08:07:02.200 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:07:02.200 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:07:02.200 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:admin_socket: invalid command 2021-02-20T08:07:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T08:07:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:2 2021-02-20T08:07:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:07:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:07:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T08:07:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:2 2021-02-20T08:07:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:07:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:07:02.205 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:07:02.205 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:07:02.205 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:07:02.205 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:07:02.205 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:07:02.206 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=2 2021-02-20T08:07:02.206 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:07:02.206 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T08:07:02.206 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:2 2021-02-20T08:07:02.207 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:07:02.207 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:07:02.208 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T08:07:02.208 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:2 2021-02-20T08:07:02.209 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:07:02.209 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:07:02.209 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:07:02.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:07:02.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:07:02.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:07:02.211 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:07:02.211 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=2 2021-02-20T08:07:02.211 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:07:02.212 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T08:07:02.223 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:07:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:07:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:07:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T08:07:02.321 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:07:02.321 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:07:02.322 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:admin_socket: invalid command 2021-02-20T08:07:02.324 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T08:07:02.325 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:3 2021-02-20T08:07:02.325 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:07:02.325 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:07:02.325 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T08:07:02.326 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:3 2021-02-20T08:07:02.326 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:07:02.327 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:07:02.327 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:07:02.327 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=3 2021-02-20T08:07:02.327 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T08:07:02.328 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:07:02.328 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:07:02.328 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=3 2021-02-20T08:07:02.328 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=3 2021-02-20T08:07:02.329 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T08:07:02.329 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:3 2021-02-20T08:07:02.329 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:07:02.329 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:07:02.330 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T08:07:02.330 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:3 2021-02-20T08:07:02.330 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:07:02.330 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:07:02.330 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:07:02.331 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=3 2021-02-20T08:07:02.331 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T08:07:02.331 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:07:02.331 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:07:02.331 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=3 2021-02-20T08:07:02.332 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=3 2021-02-20T08:07:02.332 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T08:07:02.344 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:07:02.344 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:07:02.345 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:07:02.345 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T08:07:02.438 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:07:02.439 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:07:02.439 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:admin_socket: invalid command 2021-02-20T08:07:02.440 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 4 2021-02-20T08:07:06.441 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:07:06.441 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ seq 0 3 2021-02-20T08:07:06.443 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:07:06.443 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:0 2021-02-20T08:07:06.444 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:07:06.444 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:07:06.444 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:07:06.444 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:07:06.445 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:07:06.445 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:07:06.445 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:07:06.445 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:07:06.445 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:07:06.446 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:07:06.446 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:07:06.446 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:07:06.446 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:07:06.446 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:07:06.447 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:07:06.447 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:07:06.447 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:07:06.447 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:07:06.448 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:07:06.448 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:07:06.448 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:07:06.448 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:07:06.448 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:07:06.449 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:07:06.449 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:07:06.449 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:07:06.449 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:07:06.449 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:07:06.450 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:07:06.464 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:07:06.464 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:07:06.465 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:07:06.465 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:07:06.580 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:07:06.581 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ status_result= "state": "Replaying" 2021-02-20T08:07:06.581 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo "state": "Replaying" 2021-02-20T08:07:06.581 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep -i Replaying 2021-02-20T08:07:06.582 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: "state": "Replaying" 2021-02-20T08:07:06.583 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_state=started 2021-02-20T08:07:06.583 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test started = started 2021-02-20T08:07:06.583 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:07:06.583 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ clean_snap_name= 2021-02-20T08:07:06.583 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ seq 1 10 2021-02-20T08:07:06.584 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T08:07:06.584 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:07:06.584 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:07:06.585 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:07:06.585 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:07:06.585 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=11 2021-02-20T08:07:06.586 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:07:06.586 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T08:07:07.021 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 59085824 bytes 2021-02-20T08:07:07.215 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 83407872 bytes 2021-02-20T08:07:07.370 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 110019584 bytes 2021-02-20T08:07:07.526 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 145900544 bytes 2021-02-20T08:07:07.634 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 166024192 bytes 2021-02-20T08:07:07.786 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 194678784 bytes 2021-02-20T08:07:07.986 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 262621184 bytes 2021-02-20T08:07:08.095 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 301947904 bytes 2021-02-20T08:07:08.351 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 362143744 bytes 2021-02-20T08:07:08.529 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 418991104 bytes 2021-02-20T08:07:08.797 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 495127552 bytes 2021-02-20T08:07:08.997 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 567528448 bytes 2021-02-20T08:07:09.178 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 632641536 bytes 2021-02-20T08:07:09.296 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 633271296 bytes 2021-02-20T08:07:09.378 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 641019904 bytes 2021-02-20T08:07:09.522 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 677173248 bytes 2021-02-20T08:07:09.695 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 719328256 bytes 2021-02-20T08:07:09.881 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 774375424 bytes 2021-02-20T08:07:10.019 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 810695680 bytes 2021-02-20T08:07:10.147 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 833592320 bytes 2021-02-20T08:07:10.281 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 867164160 bytes 2021-02-20T08:07:10.456 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 908661760 bytes 2021-02-20T08:07:10.672 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 979360768 bytes 2021-02-20T08:07:10.781 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 997134336 bytes 2021-02-20T08:07:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 1008380928 bytes 2021-02-20T08:07:10.964 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 1009027072 bytes 2021-02-20T08:07:11.050 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2700 IOs, 1025551360 bytes 2021-02-20T08:07:11.191 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2800 IOs, 1047658496 bytes 2021-02-20T08:07:11.364 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2900 IOs, 1091490816 bytes 2021-02-20T08:07:11.481 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3000 IOs, 1109867520 bytes 2021-02-20T08:07:11.637 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3100 IOs, 1159384064 bytes 2021-02-20T08:07:11.838 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3200 IOs, 1200881664 bytes 2021-02-20T08:07:11.955 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3300 IOs, 1212567552 bytes 2021-02-20T08:07:12.066 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3400 IOs, 1231723520 bytes 2021-02-20T08:07:12.229 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3500 IOs, 1270193152 bytes 2021-02-20T08:07:12.336 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3600 IOs, 1287063552 bytes 2021-02-20T08:07:12.551 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3700 IOs, 1354418176 bytes 2021-02-20T08:07:12.723 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3800 IOs, 1432836096 bytes 2021-02-20T08:07:12.868 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3900 IOs, 1433434112 bytes 2021-02-20T08:07:13.044 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4000 IOs, 1478955008 bytes 2021-02-20T08:07:13.203 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4100 IOs, 1509640192 bytes 2021-02-20T08:07:13.442 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4200 IOs, 1592879104 bytes 2021-02-20T08:07:13.608 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4300 IOs, 1635126272 bytes 2021-02-20T08:07:13.778 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4400 IOs, 1679493120 bytes 2021-02-20T08:07:13.971 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4500 IOs, 1725924352 bytes 2021-02-20T08:07:14.183 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4600 IOs, 1788259328 bytes 2021-02-20T08:07:14.324 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4700 IOs, 1830725632 bytes 2021-02-20T08:07:14.505 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4800 IOs, 1871510528 bytes 2021-02-20T08:07:14.668 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4900 IOs, 1903064064 bytes 2021-02-20T08:07:14.827 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5000 IOs, 1943301120 bytes 2021-02-20T08:07:14.960 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5100 IOs, 1968793600 bytes 2021-02-20T08:07:15.105 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5200 IOs, 1998545920 bytes 2021-02-20T08:07:15.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5300 IOs, 2018536448 bytes 2021-02-20T08:07:15.385 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5400 IOs, 2055296000 bytes 2021-02-20T08:07:15.517 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5500 IOs, 2078985216 bytes 2021-02-20T08:07:15.615 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5600 IOs, 2094081024 bytes 2021-02-20T08:07:15.805 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5700 IOs, 2129966080 bytes 2021-02-20T08:07:16.041 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5800 IOs, 2195762176 bytes 2021-02-20T08:07:16.150 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5900 IOs, 2210099200 bytes 2021-02-20T08:07:16.268 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:6000 IOs, 2240031744 bytes 2021-02-20T08:07:16.420 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:6100 IOs, 2265283584 bytes 2021-02-20T08:07:16.522 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:6200 IOs, 2272799744 bytes 2021-02-20T08:07:16.666 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:6300 IOs, 2303330304 bytes 2021-02-20T08:07:16.803 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:6400 IOs, 2331620352 bytes 2021-02-20T08:07:17.010 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:6500 IOs, 2391624704 bytes 2021-02-20T08:07:17.133 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:6600 IOs, 2411503616 bytes 2021-02-20T08:07:17.311 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:6700 IOs, 2456822784 bytes 2021-02-20T08:07:17.428 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:6800 IOs, 2474735616 bytes 2021-02-20T08:07:17.560 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:6900 IOs, 2501757952 bytes 2021-02-20T08:07:17.590 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:07:17.591 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:07:17.591 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:07:17.591 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:07:17.592 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:07:17.592 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:07:17.593 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:07:17.593 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:07:17.593 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:07:17.594 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local description_pattern= 2021-02-20T08:07:17.594 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local service_pattern= 2021-02-20T08:07:17.595 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:07:18.594 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:07:18.595 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:07:18.595 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:07:18.596 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:07:18.596 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:07:18.596 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local description_pattern= 2021-02-20T08:07:18.597 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local service_pattern= 2021-02-20T08:07:18.597 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T08:07:18.597 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T08:07:18.598 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:07:18.598 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:07:18.598 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T08:07:18.598 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ tee /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:07:18.665 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:test: 2021-02-20T08:07:18.666 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: global_id: 3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd 2021-02-20T08:07:18.666 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state: up+replaying 2021-02-20T08:07:18.666 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"remote_snapshot_timestamp":1613808416,"replay_state":"idle"} 2021-02-20T08:07:18.666 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: service: mirror.0 on smithi090 2021-02-20T08:07:18.666 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: last_update: 2021-02-20 08:07:06 2021-02-20T08:07:18.667 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: peer_sites: 2021-02-20T08:07:18.667 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: name: cluster2 2021-02-20T08:07:18.667 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state: up+stopped 2021-02-20T08:07:18.667 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: description: local image is primary 2021-02-20T08:07:18.668 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: last_update: 2021-02-20 08:07:06 2021-02-20T08:07:18.675 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:07:18.679 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: state: up+replaying 2021-02-20T08:07:18.679 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"remote_snapshot_timestamp":1613808416,"replay_state":"idle"} 2021-02-20T08:07:18.680 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^ description: .* /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:07:18.682 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:07:18.682 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo up+replaying 2021-02-20T08:07:18.683 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^up+ 2021-02-20T08:07:18.683 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep service: *mirror..* on /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:07:18.684 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:up+replaying 2021-02-20T08:07:18.685 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: service: mirror.0 on smithi090 2021-02-20T08:07:18.685 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:07:18.690 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local last_update=2021-02-20 08:07:06 2021-02-20T08:07:18.691 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 08:07:06 2021-02-20T08:07:18.691 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:07:18.691 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:07:18.692 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:07:18.692 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:07:18.693 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local prev_last_update=2021-02-20 08:07:06 2021-02-20T08:07:18.693 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T08:07:18.694 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T08:07:18.694 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:07:18.695 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror.mirror_status 2021-02-20T08:07:18.695 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T08:07:18.732 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T08:07:18.787 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local last_update state 2021-02-20T08:07:18.788 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T08:07:18.812 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ last_update=2021-02-20 08:07:06 2021-02-20T08:07:18.812 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T08:07:18.816 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ state=up+replaying 2021-02-20T08:07:18.816 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo up+replaying+ 2021-02-20T08:07:18.817 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:grep up+replaying 2021-02-20T08:07:18.818 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:up+replaying 2021-02-20T08:07:18.819 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:07:18.819 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:07:18.820 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap1 2021-02-20T08:07:18.820 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_snap cluster2 mirror test snap1 2021-02-20T08:07:18.821 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:07:18.821 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:07:18.821 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:07:18.822 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap1 2021-02-20T08:07:18.822 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap1 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:07:26.010 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8705064531267924) > weight (0.25) 2021-02-20T08:07:26.016 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.48442222510490696) > weight (0.25) 2021-02-20T08:07:26.016 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5791441248876104) > weight (0.25) 2021-02-20T08:07:26.017 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T08:07:26.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T08:07:26.019 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T08:07:26.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T08:07:26.016+0000 7f75ce879700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 18982) UID: 1000 2021-02-20T08:07:26.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:07:29.011 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T08:07:29.012 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T08:07:29.012 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T08:07:29.013 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T08:07:29.012+0000 7fcd4fe10700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 18993) UID: 1000 2021-02-20T08:07:29.213 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:07:33.020 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T08:07:33.020 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:07:33.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:07:33.021 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T08:07:33.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T08:07:33.022 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T08:07:33.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T08:07:33.025 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T08:07:34.013 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T08:07:34.014 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:07:34.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:07:34.014 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T08:07:34.015 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T08:07:34.015 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T08:07:34.018 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T08:07:34.018 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T08:07:48.900 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T08:07:48.901 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:07:48.901 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:07:48.902 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:07:48.902 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T08:07:48.902 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:07:48.902 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:07:48.903 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:07:48.903 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state=started 2021-02-20T08:07:48.903 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:07:48.903 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:07:49.901 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T08:07:49.902 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:07:49.902 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:07:49.902 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:07:49.903 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local test_state=started 2021-02-20T08:07:49.903 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_result 2021-02-20T08:07:49.903 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_state=stopped 2021-02-20T08:07:49.903 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T08:07:49.904 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_instance=cluster1 2021-02-20T08:07:49.904 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:07:49.904 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:07:49.904 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance=cluster1 2021-02-20T08:07:49.904 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loop_instance 2021-02-20T08:07:49.905 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ + sleep 0 2021-02-20T08:07:49.905 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:grep -i state 2021-02-20T08:07:49.907 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:07:49.907 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ seq 0 3 2021-02-20T08:07:49.908 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:07:49.908 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:0 2021-02-20T08:07:49.908 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:07:49.909 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:07:49.909 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:07:49.909 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:07:49.909 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:07:49.909 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:07:49.910 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:07:49.910 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:07:49.911 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:07:49.911 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:07:49.912 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:07:49.912 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:07:49.912 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:07:49.912 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:07:49.913 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:07:49.913 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:07:49.913 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:07:49.914 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:07:49.914 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:07:49.914 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:07:49.914 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:07:49.915 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:07:49.915 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:07:49.915 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:07:49.915 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:07:49.916 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:07:49.916 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:07:49.916 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:07:49.917 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:07:49.921 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:07:49.921 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:07:49.922 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:07:49.922 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:07:50.033 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:07:50.042 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ status_result= "state": "Replaying" 2021-02-20T08:07:50.044 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ + echo "state": "Replaying" 2021-02-20T08:07:50.044 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:grep -i Replaying 2021-02-20T08:07:50.045 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_state=started 2021-02-20T08:07:50.045 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test started = started 2021-02-20T08:07:50.046 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:07:50.046 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T08:07:50.047 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:07:50.047 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:07:50.047 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:07:50.048 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:07:50.048 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = journal ] 2021-02-20T08:07:50.050 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = snapshot ] 2021-02-20T08:07:50.050 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T08:07:50.050 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:07:50.051 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:07:50.051 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:07:50.051 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:07:50.052 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T08:07:50.052 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster2-mirror-test.status 2021-02-20T08:07:50.052 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:07:50.053 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:07:50.053 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T08:07:50.053 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror-test.status 2021-02-20T08:07:50.054 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:07:50.054 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:07:50.055 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T08:07:50.055 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:07:50.055 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:07:50.055 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:07:50.056 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T08:07:50.056 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: "state": "Replaying" 2021-02-20T08:07:50.913 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Snapshot ID: 6 2021-02-20T08:07:50.926 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:07:50.926 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:07:50.926 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:07:50.927 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:07:50.927 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:07:50.927 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T08:07:50.927 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:07:50.980 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T08:07:50.983 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snapshot_id=6 2021-02-20T08:07:50.983 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:07:50.983 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.2 2021-02-20T08:07:51.185 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:07:51.185 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:07:51.185 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:07:51.185 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:07:51.186 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:07:51.186 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:07:51.186 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:07:51.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:07:51.260 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=4 2021-02-20T08:07:51.260 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 6 = 4 2021-02-20T08:07:51.260 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.4 2021-02-20T08:07:51.663 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:07:51.663 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:07:51.664 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:07:51.664 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:07:51.664 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:07:51.665 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:07:51.665 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:07:51.730 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:07:51.735 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=4 2021-02-20T08:07:51.735 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 6 = 4 2021-02-20T08:07:51.736 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.8 2021-02-20T08:07:52.537 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:07:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:07:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:07:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:07:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:07:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:07:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:07:52.608 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:07:52.608 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:07:52.612 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:07:52.613 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:07:52.613 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:07:52.614 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:07:52.614 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 6 = 2021-02-20T08:07:52.614 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1.6 2021-02-20T08:07:54.215 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:07:54.215 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:07:54.215 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:07:54.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:07:54.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:07:54.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:07:54.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:07:54.285 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:07:54.291 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=6 2021-02-20T08:07:54.291 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 6 = 6 2021-02-20T08:07:54.291 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:07:54.291 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snap_present cluster1 mirror test snap1 2021-02-20T08:07:54.292 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:07:54.292 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:07:54.292 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:07:54.292 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap1 2021-02-20T08:07:54.292 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:07:54.293 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:07:55.292 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 info mirror/test@snap1 2021-02-20T08:07:55.361 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:rbd image 'test': 2021-02-20T08:07:55.362 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: size 512 MiB in 128 objects 2021-02-20T08:07:55.362 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: order 22 (4 MiB objects) 2021-02-20T08:07:55.363 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: snapshot_count: 3 2021-02-20T08:07:55.364 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: id: 10a0aa806784 2021-02-20T08:07:55.364 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: block_name_prefix: rbd_data.10a0aa806784 2021-02-20T08:07:55.364 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: format: 2 2021-02-20T08:07:55.365 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: features: layering, exclusive-lock, object-map, fast-diff, operations, non-primary 2021-02-20T08:07:55.365 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: op_features: snap-trash 2021-02-20T08:07:55.365 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: flags: 2021-02-20T08:07:55.365 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: create_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:07:55.366 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: access_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:07:55.366 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: modify_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:07:55.366 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: protected: False 2021-02-20T08:07:55.366 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring state: enabled 2021-02-20T08:07:55.367 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring mode: snapshot 2021-02-20T08:07:55.367 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring global id: 3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd 2021-02-20T08:07:55.367 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring primary: false 2021-02-20T08:07:55.369 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:07:55.369 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:07:55.369 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap1 2021-02-20T08:07:55.370 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:07:55.370 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:07:55.371 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap1 2021-02-20T08:07:55.371 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:07:55.371 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:07:55.371 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:07:55.372 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:07:55.372 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1 /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:07:56.034 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:56.038 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1 /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:07:56.871 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:56.875 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:07:57.151 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:07:57.278 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:07:57.278 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ clean_snap_name=snap1-clean 2021-02-20T08:07:57.279 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_snap cluster2 mirror test snap1-clean 2021-02-20T08:07:57.279 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:07:57.279 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:07:57.279 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:07:57.280 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap1-clean 2021-02-20T08:07:57.280 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap1-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T08:07:57.941 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T08:07:57.942 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:07:57.942 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:07:57.942 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:07:57.942 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:07:57.943 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=14 2021-02-20T08:07:57.943 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:07:57.944 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T08:07:58.476 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 55952384 bytes 2021-02-20T08:07:58.755 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 113697792 bytes 2021-02-20T08:07:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 129034240 bytes 2021-02-20T08:07:59.060 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 169060352 bytes 2021-02-20T08:07:59.266 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 232496128 bytes 2021-02-20T08:07:59.518 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 286903296 bytes 2021-02-20T08:07:59.663 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 312099840 bytes 2021-02-20T08:07:59.874 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 370949120 bytes 2021-02-20T08:08:00.108 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 443762688 bytes 2021-02-20T08:08:00.346 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 514777088 bytes 2021-02-20T08:08:00.533 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 567373824 bytes 2021-02-20T08:08:00.667 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 586670080 bytes 2021-02-20T08:08:00.855 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 635874304 bytes 2021-02-20T08:08:00.976 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 669714432 bytes 2021-02-20T08:08:01.219 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 736992256 bytes 2021-02-20T08:08:01.421 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 772163584 bytes 2021-02-20T08:08:01.549 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 792017920 bytes 2021-02-20T08:08:01.757 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 850620416 bytes 2021-02-20T08:08:01.881 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 886863872 bytes 2021-02-20T08:08:02.008 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 888316928 bytes 2021-02-20T08:08:02.168 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 918583296 bytes 2021-02-20T08:08:02.470 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 1015417856 bytes 2021-02-20T08:08:02.636 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 1047422976 bytes 2021-02-20T08:08:02.831 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 1102246912 bytes 2021-02-20T08:08:03.054 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 1161931776 bytes 2021-02-20T08:08:03.311 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 1236822016 bytes 2021-02-20T08:08:03.498 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2700 IOs, 1280515072 bytes 2021-02-20T08:08:03.656 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2800 IOs, 1312501760 bytes 2021-02-20T08:08:03.908 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2900 IOs, 1388595200 bytes 2021-02-20T08:08:04.101 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3000 IOs, 1435144192 bytes 2021-02-20T08:08:04.417 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3100 IOs, 1537676288 bytes 2021-02-20T08:08:04.537 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3200 IOs, 1559399424 bytes 2021-02-20T08:08:04.782 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3300 IOs, 1627114496 bytes 2021-02-20T08:08:05.058 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3400 IOs, 1708921856 bytes 2021-02-20T08:08:05.269 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3500 IOs, 1795381248 bytes 2021-02-20T08:08:05.524 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3600 IOs, 1837125632 bytes 2021-02-20T08:08:05.690 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3700 IOs, 1880595456 bytes 2021-02-20T08:08:05.888 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3800 IOs, 1925401600 bytes 2021-02-20T08:08:06.042 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3900 IOs, 1956826112 bytes 2021-02-20T08:08:06.182 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4000 IOs, 1975986176 bytes 2021-02-20T08:08:06.397 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4100 IOs, 2035850240 bytes 2021-02-20T08:08:06.556 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4200 IOs, 2071387136 bytes 2021-02-20T08:08:06.718 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4300 IOs, 2104871936 bytes 2021-02-20T08:08:07.008 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4400 IOs, 2221718528 bytes 2021-02-20T08:08:07.323 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4500 IOs, 2300212224 bytes 2021-02-20T08:08:07.456 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4600 IOs, 2318533632 bytes 2021-02-20T08:08:07.671 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4700 IOs, 2378717184 bytes 2021-02-20T08:08:08.002 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4800 IOs, 2489546752 bytes 2021-02-20T08:08:08.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4900 IOs, 2514870272 bytes 2021-02-20T08:08:08.385 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5000 IOs, 2586776576 bytes 2021-02-20T08:08:08.618 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5100 IOs, 2647401472 bytes 2021-02-20T08:08:08.769 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5200 IOs, 2673829888 bytes 2021-02-20T08:08:08.913 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5300 IOs, 2705409024 bytes 2021-02-20T08:08:09.212 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5400 IOs, 2794033152 bytes 2021-02-20T08:08:09.352 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5500 IOs, 2817344512 bytes 2021-02-20T08:08:09.528 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5600 IOs, 2855439360 bytes 2021-02-20T08:08:09.741 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5700 IOs, 2919087104 bytes 2021-02-20T08:08:10.158 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5800 IOs, 3064157184 bytes 2021-02-20T08:08:10.378 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5900 IOs, 3136433152 bytes 2021-02-20T08:08:10.519 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:6000 IOs, 3137049600 bytes 2021-02-20T08:08:10.810 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:6100 IOs, 3224038400 bytes 2021-02-20T08:08:10.998 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:6200 IOs, 3267776512 bytes 2021-02-20T08:08:11.171 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:6300 IOs, 3306565632 bytes 2021-02-20T08:08:11.448 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:6400 IOs, 3415377920 bytes 2021-02-20T08:08:11.603 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:6500 IOs, 3415968768 bytes 2021-02-20T08:08:11.725 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:6600 IOs, 3430077440 bytes 2021-02-20T08:08:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:6700 IOs, 3449860096 bytes 2021-02-20T08:08:11.946 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:08:11.947 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:08:11.947 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:08:11.948 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:08:11.948 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:08:11.949 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:08:11.949 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:08:11.949 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:08:11.950 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:08:11.950 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local description_pattern= 2021-02-20T08:08:11.951 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local service_pattern= 2021-02-20T08:08:11.951 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:08:12.947 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:08:12.948 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:08:12.948 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:08:12.948 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:08:12.948 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:08:12.949 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local description_pattern= 2021-02-20T08:08:12.949 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local service_pattern= 2021-02-20T08:08:12.949 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T08:08:12.949 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T08:08:12.949 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:08:12.950 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:08:12.950 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T08:08:12.951 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ tee /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:08:13.021 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:test: 2021-02-20T08:08:13.022 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: global_id: 3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd 2021-02-20T08:08:13.022 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state: up+replaying 2021-02-20T08:08:13.022 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: description: replaying, {"bytes_per_second":17335193.6,"bytes_per_snapshot":260027904.0,"local_snapshot_timestamp":1613808470,"remote_snapshot_timestamp":1613808470,"replay_state":"idle"} 2021-02-20T08:08:13.023 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: service: mirror.0 on smithi090 2021-02-20T08:08:13.023 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: last_update: 2021-02-20 08:08:03 2021-02-20T08:08:13.023 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: peer_sites: 2021-02-20T08:08:13.023 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: name: cluster2 2021-02-20T08:08:13.023 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state: up+stopped 2021-02-20T08:08:13.024 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: description: local image is primary 2021-02-20T08:08:13.024 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: last_update: 2021-02-20 08:08:04 2021-02-20T08:08:13.031 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:08:13.033 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: state: up+replaying 2021-02-20T08:08:13.033 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^ description: .* /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:08:13.036 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: description: replaying, {"bytes_per_second":17335193.6,"bytes_per_snapshot":260027904.0,"local_snapshot_timestamp":1613808470,"remote_snapshot_timestamp":1613808470,"replay_state":"idle"} 2021-02-20T08:08:13.037 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:up+replaying 2021-02-20T08:08:13.037 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:08:13.037 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ + grep ^up+ 2021-02-20T08:08:13.037 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:echo up+replaying 2021-02-20T08:08:13.038 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep service: *mirror..* on /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:08:13.038 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: service: mirror.0 on smithi090 2021-02-20T08:08:13.039 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:08:13.040 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local last_update=2021-02-20 08:08:03 2021-02-20T08:08:13.041 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 08:08:03 2021-02-20T08:08:13.041 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:08:13.041 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:08:13.041 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:08:13.041 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:08:13.042 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local prev_last_update=2021-02-20 08:08:03 2021-02-20T08:08:13.042 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T08:08:13.043 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T08:08:13.043 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:08:13.043 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror.mirror_status 2021-02-20T08:08:13.044 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T08:08:13.090 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T08:08:13.133 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local last_update state 2021-02-20T08:08:13.134 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T08:08:13.139 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ last_update=2021-02-20 08:08:03 2021-02-20T08:08:13.139 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T08:08:13.145 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ state=up+replaying 2021-02-20T08:08:13.145 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo up+replaying 2021-02-20T08:08:13.145 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep up+replaying 2021-02-20T08:08:13.146 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:up+replaying 2021-02-20T08:08:13.146 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:08:13.147 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:08:13.147 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap2 2021-02-20T08:08:13.147 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_snap cluster2 mirror test snap2 2021-02-20T08:08:13.147 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:08:13.148 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:08:13.148 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:08:13.148 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap2 2021-02-20T08:08:13.148 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap2 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:08:43.909 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T08:08:43.909 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:08:43.909 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:08:43.910 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:08:43.910 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T08:08:43.910 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:08:43.910 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:08:43.911 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:08:43.911 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state=started 2021-02-20T08:08:43.911 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:08:43.911 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:08:44.910 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T08:08:44.910 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:08:44.910 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:08:44.911 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:08:44.911 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local test_state=started 2021-02-20T08:08:44.911 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_result 2021-02-20T08:08:44.911 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_state=stopped 2021-02-20T08:08:44.911 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T08:08:44.912 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_instance=cluster1 2021-02-20T08:08:44.912 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:08:44.912 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:08:44.912 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ + local instance=cluster1 2021-02-20T08:08:44.913 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loop_instance 2021-02-20T08:08:44.913 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0 2021-02-20T08:08:44.913 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:grep -i state 2021-02-20T08:08:44.915 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:08:44.915 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ seq 0 3 2021-02-20T08:08:44.916 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:08:44.916 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:0 2021-02-20T08:08:44.916 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:08:44.917 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:08:44.917 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:08:44.917 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:08:44.917 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:08:44.918 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:08:44.919 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:08:44.919 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:08:44.919 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:08:44.919 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:08:44.920 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:08:44.920 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:08:44.920 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:08:44.921 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:08:44.921 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:08:44.921 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:08:44.921 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:08:44.921 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:08:44.922 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:08:44.922 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:08:44.922 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:08:44.922 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:08:44.923 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:08:44.923 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:08:44.923 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:08:44.923 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:08:44.924 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:08:44.924 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:08:44.924 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:08:44.937 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:08:44.937 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:08:44.937 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:08:44.937 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:08:45.051 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:08:45.052 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ status_result= "state": "Replaying" 2021-02-20T08:08:45.052 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo "state": "Replaying" 2021-02-20T08:08:45.053 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep -i Replaying 2021-02-20T08:08:45.055 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: "state": "Replaying" 2021-02-20T08:08:45.055 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_state=started 2021-02-20T08:08:45.055 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test started = started 2021-02-20T08:08:45.055 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:08:45.056 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T08:08:45.056 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:08:45.056 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:08:45.056 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:08:45.057 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:08:45.057 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = journal ] 2021-02-20T08:08:45.057 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = snapshot ] 2021-02-20T08:08:45.058 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T08:08:45.058 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:08:45.058 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:08:45.058 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:08:45.059 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:08:45.059 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T08:08:45.059 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster2-mirror-test.status 2021-02-20T08:08:45.059 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:08:45.060 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:08:45.061 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T08:08:45.061 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror-test.status 2021-02-20T08:08:45.061 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:08:45.061 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:08:45.062 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T08:08:45.062 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:08:45.062 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:08:45.062 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:08:45.063 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T08:08:45.928 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Snapshot ID: 10 2021-02-20T08:08:45.946 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:08:45.946 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:08:45.946 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:08:45.947 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:08:45.947 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:08:45.947 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T08:08:45.947 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:08:46.003 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T08:08:46.008 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snapshot_id=10 2021-02-20T08:08:46.008 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:08:46.009 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.2 2021-02-20T08:08:46.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:08:46.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:08:46.211 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:08:46.211 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:08:46.211 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:08:46.211 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:08:46.211 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:08:46.277 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:08:46.282 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=6 2021-02-20T08:08:46.282 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 10 = 6 2021-02-20T08:08:46.282 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.4 2021-02-20T08:08:46.684 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:08:46.684 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:08:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:08:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:08:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:08:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:08:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:08:46.756 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:08:46.762 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=6 2021-02-20T08:08:46.762 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 10 = 6 2021-02-20T08:08:46.762 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.8 2021-02-20T08:08:47.564 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:08:47.564 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:08:47.564 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:08:47.565 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:08:47.565 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:08:47.565 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:08:47.565 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:08:47.634 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:08:47.635 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:08:47.639 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:08:47.639 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:08:47.639 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:08:47.640 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:08:47.640 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 10 = 2021-02-20T08:08:47.641 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1.6 2021-02-20T08:08:49.241 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:08:49.241 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:08:49.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:08:49.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:08:49.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:08:49.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:08:49.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:08:49.316 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:08:49.316 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:08:49.319 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:08:49.320 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:08:49.320 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:08:49.321 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:08:49.321 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 10 = 2021-02-20T08:08:49.321 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 2 2021-02-20T08:08:51.322 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:08:51.322 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:08:51.322 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:08:51.322 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:08:51.323 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:08:51.323 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:08:51.323 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:08:51.391 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:08:51.397 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=10 2021-02-20T08:08:51.397 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 10 = 10 2021-02-20T08:08:51.398 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:08:51.398 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snap_present cluster1 mirror test snap2 2021-02-20T08:08:51.399 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:08:51.399 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:08:51.400 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:08:51.400 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap2 2021-02-20T08:08:51.400 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:08:51.400 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:08:52.398 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 info mirror/test@snap2 2021-02-20T08:08:52.468 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:rbd image 'test': 2021-02-20T08:08:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: size 512 MiB in 128 objects 2021-02-20T08:08:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: order 22 (4 MiB objects) 2021-02-20T08:08:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: snapshot_count: 4 2021-02-20T08:08:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: id: 10a0aa806784 2021-02-20T08:08:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: block_name_prefix: rbd_data.10a0aa806784 2021-02-20T08:08:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: format: 2 2021-02-20T08:08:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: features: layering, exclusive-lock, object-map, fast-diff, non-primary 2021-02-20T08:08:52.471 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: op_features: 2021-02-20T08:08:52.471 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: flags: 2021-02-20T08:08:52.471 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: create_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:08:52.471 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: access_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:08:52.471 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: modify_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:08:52.472 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: protected: False 2021-02-20T08:08:52.472 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring state: enabled 2021-02-20T08:08:52.472 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring mode: snapshot 2021-02-20T08:08:52.472 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring global id: 3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd 2021-02-20T08:08:52.472 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring primary: false 2021-02-20T08:08:52.474 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:08:52.474 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snap1-clean ] 2021-02-20T08:08:52.474 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap1-clean 2021-02-20T08:08:52.475 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:08:52.475 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:08:52.475 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap1-clean 2021-02-20T08:08:52.475 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:08:52.475 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:08:52.476 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:08:52.476 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:08:52.476 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1-clean /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:08:53.122 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:08:53.127 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1-clean /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:08:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:08:53.886 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:08:54.159 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:08:54.279 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:08:54.279 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap2 2021-02-20T08:08:54.279 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:08:54.279 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:08:54.280 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap2 2021-02-20T08:08:54.280 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:08:54.280 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:08:54.280 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:08:54.281 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:08:54.281 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2 /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:08:54.939 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:08:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2 /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:08:55.825 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:08:55.831 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:08:56.111 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:08:56.238 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:08:56.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ clean_snap_name=snap2-clean 2021-02-20T08:08:56.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_snap cluster2 mirror test snap2-clean 2021-02-20T08:08:56.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:08:56.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:08:56.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:08:56.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap2-clean 2021-02-20T08:08:56.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap2-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T08:08:56.965 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T08:08:56.966 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:08:56.966 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:08:56.966 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:08:56.967 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:08:56.967 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=14 2021-02-20T08:08:56.968 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:08:56.968 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T08:08:57.501 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 26351616 bytes 2021-02-20T08:08:58.486 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 81757184 bytes 2021-02-20T08:08:58.744 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 145016832 bytes 2021-02-20T08:08:58.953 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 184689664 bytes 2021-02-20T08:08:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 219556864 bytes 2021-02-20T08:08:59.341 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 275863552 bytes 2021-02-20T08:08:59.462 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 279569408 bytes 2021-02-20T08:08:59.643 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 313805824 bytes 2021-02-20T08:08:59.949 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 413315072 bytes 2021-02-20T08:09:00.124 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 444313600 bytes 2021-02-20T08:09:00.282 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 474191872 bytes 2021-02-20T08:09:00.412 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 490169344 bytes 2021-02-20T08:09:00.561 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 515858432 bytes 2021-02-20T08:09:00.798 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 575049728 bytes 2021-02-20T08:09:00.942 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 598554624 bytes 2021-02-20T08:09:01.243 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 635440128 bytes 2021-02-20T08:09:01.585 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 768538624 bytes 2021-02-20T08:09:01.725 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 770289664 bytes 2021-02-20T08:09:01.869 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 803207168 bytes 2021-02-20T08:09:02.107 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 858729472 bytes 2021-02-20T08:09:02.205 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 879892480 bytes 2021-02-20T08:09:02.491 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 944721920 bytes 2021-02-20T08:09:02.713 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 998492160 bytes 2021-02-20T08:09:02.836 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 1012066304 bytes 2021-02-20T08:09:03.011 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 1050901504 bytes 2021-02-20T08:09:03.235 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 1119123456 bytes 2021-02-20T08:09:03.469 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2700 IOs, 1167552512 bytes 2021-02-20T08:09:03.769 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2800 IOs, 1250748416 bytes 2021-02-20T08:09:03.991 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2900 IOs, 1309660160 bytes 2021-02-20T08:09:04.207 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3000 IOs, 1360504832 bytes 2021-02-20T08:09:04.391 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3100 IOs, 1399532544 bytes 2021-02-20T08:09:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3200 IOs, 1443564544 bytes 2021-02-20T08:09:04.838 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3300 IOs, 1512180736 bytes 2021-02-20T08:09:04.996 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3400 IOs, 1540379648 bytes 2021-02-20T08:09:05.125 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3500 IOs, 1553781760 bytes 2021-02-20T08:09:05.326 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3600 IOs, 1595954176 bytes 2021-02-20T08:09:05.506 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3700 IOs, 1628878848 bytes 2021-02-20T08:09:05.671 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3800 IOs, 1657548800 bytes 2021-02-20T08:09:05.799 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3900 IOs, 1667726336 bytes 2021-02-20T08:09:05.984 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4000 IOs, 1707573248 bytes 2021-02-20T08:09:06.140 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4100 IOs, 1732316160 bytes 2021-02-20T08:09:06.265 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4200 IOs, 1743789056 bytes 2021-02-20T08:09:06.458 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4300 IOs, 1789821952 bytes 2021-02-20T08:09:06.790 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4400 IOs, 1897359360 bytes 2021-02-20T08:09:06.977 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4500 IOs, 1931145216 bytes 2021-02-20T08:09:07.026 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2516142559892748) > weight (0.25) 2021-02-20T08:09:07.026 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7247655719718029) > weight (0.25) 2021-02-20T08:09:07.027 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8470456428000747) > weight (0.25) 2021-02-20T08:09:07.027 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9908821524194593) > weight (0.25) 2021-02-20T08:09:07.027 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T08:09:07.322 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4600 IOs, 2053805056 bytes 2021-02-20T08:09:07.522 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4700 IOs, 2103587840 bytes 2021-02-20T08:09:07.630 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4800 IOs, 2107459584 bytes 2021-02-20T08:09:07.841 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4900 IOs, 2155279360 bytes 2021-02-20T08:09:08.099 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5000 IOs, 2224117760 bytes 2021-02-20T08:09:08.333 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5100 IOs, 2283937792 bytes 2021-02-20T08:09:08.577 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5200 IOs, 2339343360 bytes 2021-02-20T08:09:08.867 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5300 IOs, 2360859648 bytes 2021-02-20T08:09:09.056 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5400 IOs, 2405201920 bytes 2021-02-20T08:09:09.269 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5500 IOs, 2450876416 bytes 2021-02-20T08:09:09.492 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5600 IOs, 2513415168 bytes 2021-02-20T08:09:09.639 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5700 IOs, 2527900672 bytes 2021-02-20T08:09:09.799 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5800 IOs, 2550384640 bytes 2021-02-20T08:09:10.081 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:5900 IOs, 2629570560 bytes 2021-02-20T08:09:10.265 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:6000 IOs, 2666826752 bytes 2021-02-20T08:09:10.513 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:6100 IOs, 2759096320 bytes 2021-02-20T08:09:10.729 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:6200 IOs, 2782059520 bytes 2021-02-20T08:09:10.971 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:09:10.972 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:09:10.972 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:09:10.972 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:09:10.972 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:09:10.973 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:09:10.973 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:09:10.973 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:09:10.973 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:09:10.973 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local description_pattern= 2021-02-20T08:09:10.974 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local service_pattern= 2021-02-20T08:09:10.974 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:09:11.973 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:09:11.973 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:09:11.974 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:09:11.974 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:09:11.974 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:09:11.974 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local description_pattern= 2021-02-20T08:09:11.974 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local service_pattern= 2021-02-20T08:09:11.975 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T08:09:11.975 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T08:09:11.975 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:09:11.976 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:09:11.977 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_ARGS= rbd+ --cluster cluster1 mirror image status mirror/test 2021-02-20T08:09:11.977 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:tee /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:09:12.049 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:test: 2021-02-20T08:09:12.049 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: global_id: 3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd 2021-02-20T08:09:12.049 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state: up+replaying 2021-02-20T08:09:12.050 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: description: replaying, {"bytes_per_second":17631880.53,"bytes_per_snapshot":524506112.0,"local_snapshot_timestamp":1613808525,"remote_snapshot_timestamp":1613808525,"replay_state":"idle"} 2021-02-20T08:09:12.050 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: service: mirror.0 on smithi090 2021-02-20T08:09:12.050 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: last_update: 2021-02-20 08:09:03 2021-02-20T08:09:12.050 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: peer_sites: 2021-02-20T08:09:12.051 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: name: cluster2 2021-02-20T08:09:12.051 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state: up+stopped 2021-02-20T08:09:12.051 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: description: local image is primary 2021-02-20T08:09:12.051 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: last_update: 2021-02-20 08:09:04 2021-02-20T08:09:12.058 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:09:12.060 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: state: up+replaying 2021-02-20T08:09:12.060 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^ description: .* /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:09:12.061 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: description: replaying, {"bytes_per_second":17631880.53,"bytes_per_snapshot":524506112.0,"local_snapshot_timestamp":1613808525,"remote_snapshot_timestamp":1613808525,"replay_state":"idle"} 2021-02-20T08:09:12.062 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:09:12.062 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo up+replaying 2021-02-20T08:09:12.063 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^up+ 2021-02-20T08:09:12.063 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:up+replaying 2021-02-20T08:09:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep service: *mirror..* on /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:09:12.065 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: service: mirror.0 on smithi090 2021-02-20T08:09:12.065 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:09:12.068 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local last_update=2021-02-20 08:09:03 2021-02-20T08:09:12.068 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 08:09:03 2021-02-20T08:09:12.068 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:09:12.069 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:09:12.069 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:09:12.069 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:09:12.069 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local prev_last_update=2021-02-20 08:09:03 2021-02-20T08:09:12.070 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T08:09:12.071 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T08:09:12.071 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:09:12.071 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror.mirror_status 2021-02-20T08:09:12.071 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T08:09:12.116 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T08:09:12.168 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local last_update state 2021-02-20T08:09:12.168 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T08:09:12.174 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ last_update=2021-02-20 08:09:03 2021-02-20T08:09:12.174 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T08:09:12.179 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ state=up+replaying 2021-02-20T08:09:12.180 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo up+replaying 2021-02-20T08:09:12.180 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep up+replaying 2021-02-20T08:09:12.181 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:up+replaying 2021-02-20T08:09:12.182 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:09:12.182 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:09:12.182 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap3 2021-02-20T08:09:12.183 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_snap cluster2 mirror test snap3 2021-02-20T08:09:12.183 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:09:12.183 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:09:12.183 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:09:12.183 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap3 2021-02-20T08:09:12.184 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap3 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:09:22.019 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9505447930897118) > weight (0.25) 2021-02-20T08:09:22.019 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T08:09:22.020 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T08:09:22.020 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T08:09:22.021 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T08:09:22.021 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:09:22.022 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T08:09:22.020+0000 7face7f21700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 19030) UID: 1000 2021-02-20T08:09:22.222 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:09:28.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:09:28.123 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T08:09:28.124 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T08:09:28.124 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T08:09:28.127 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T08:09:28.127 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T08:09:41.062 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T08:09:41.062 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:09:41.063 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:09:41.063 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:09:41.063 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T08:09:41.063 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:09:41.064 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:09:41.064 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:09:41.064 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state=started 2021-02-20T08:09:41.064 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:09:41.064 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:09:42.063 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T08:09:42.064 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:09:42.064 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:09:42.064 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:09:42.064 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local test_state=started 2021-02-20T08:09:42.065 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_result 2021-02-20T08:09:42.065 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_state=stopped 2021-02-20T08:09:42.065 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T08:09:42.065 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_instance=cluster1 2021-02-20T08:09:42.065 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:09:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ + local cluster=cluster1 2021-02-20T08:09:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance=cluster1 2021-02-20T08:09:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:grep -i+ statelocal 2021-02-20T08:09:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: loop_instance 2021-02-20T08:09:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0 2021-02-20T08:09:42.068 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:09:42.068 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ seq 0 3 2021-02-20T08:09:42.069 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:09:42.069 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:0 2021-02-20T08:09:42.069 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:09:42.069 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:09:42.070 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:09:42.070 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:09:42.070 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:09:42.070 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:09:42.070 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:09:42.071 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:09:42.071 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:09:42.071 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:09:42.071 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:09:42.072 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:09:42.072 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:09:42.072 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:09:42.072 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:09:42.072 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:09:42.073 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:09:42.073 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:09:42.073 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:09:42.073 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:09:42.074 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:09:42.075 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:09:42.075 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:09:42.075 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:09:42.075 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:09:42.075 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:09:42.076 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:09:42.076 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:09:42.076 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:09:42.088 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:09:42.088 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:09:42.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:09:42.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:09:42.206 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:09:42.206 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ status_result= "state": "Replaying" 2021-02-20T08:09:42.207 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo "state": "Replaying" 2021-02-20T08:09:42.207 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep -i Replaying 2021-02-20T08:09:42.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: "state": "Replaying" 2021-02-20T08:09:42.211 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_state=started 2021-02-20T08:09:42.211 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test started = started 2021-02-20T08:09:42.212 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:09:42.212 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T08:09:42.212 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:09:42.212 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:09:42.213 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:09:42.213 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:09:42.213 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = journal ] 2021-02-20T08:09:42.214 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = snapshot ] 2021-02-20T08:09:42.214 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T08:09:42.214 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:09:42.214 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:09:42.215 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:09:42.215 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:09:42.215 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T08:09:42.215 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster2-mirror-test.status 2021-02-20T08:09:42.215 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:09:42.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:09:42.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T08:09:42.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror-test.status 2021-02-20T08:09:42.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:09:42.217 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:09:42.217 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T08:09:42.218 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:09:42.218 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:09:42.218 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:09:42.219 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T08:09:43.071 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Snapshot ID: 14 2021-02-20T08:09:43.088 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:09:43.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:09:43.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:09:43.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:09:43.090 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:09:43.090 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T08:09:43.090 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:09:43.150 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T08:09:43.156 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snapshot_id=14 2021-02-20T08:09:43.157 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:09:43.157 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.2 2021-02-20T08:09:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:09:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:09:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:09:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:09:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:09:43.357 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:09:43.357 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:09:43.427 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:09:43.430 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=10 2021-02-20T08:09:43.431 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 14 = 10 2021-02-20T08:09:43.431 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.4 2021-02-20T08:09:43.832 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:09:43.833 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:09:43.833 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:09:43.833 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:09:43.833 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:09:43.834 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:09:43.834 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:09:43.903 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:09:43.904 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:09:43.908 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:09:43.908 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:09:43.909 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:09:43.909 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:09:43.909 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 14 = 2021-02-20T08:09:43.909 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.8 2021-02-20T08:09:44.711 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:09:44.711 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:09:44.711 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:09:44.711 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:09:44.712 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:09:44.712 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:09:44.712 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:09:44.782 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:09:44.783 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:09:44.787 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:09:44.787 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:09:44.788 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:09:44.788 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:09:44.788 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 14 = 2021-02-20T08:09:44.789 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1.6 2021-02-20T08:09:46.389 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:09:46.390 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:09:46.390 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:09:46.390 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:09:46.390 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:09:46.391 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:09:46.391 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:09:46.468 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:09:46.468 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:09:46.472 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:09:46.472 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:09:46.472 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:09:46.473 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:09:46.473 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 14 = 2021-02-20T08:09:46.473 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 2 2021-02-20T08:09:48.475 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:09:48.476 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:09:48.476 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:09:48.477 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:09:48.477 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:09:48.477 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:09:48.478 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:09:48.750 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:09:48.755 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=14 2021-02-20T08:09:48.755 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 14 = 14 2021-02-20T08:09:48.756 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:09:48.756 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snap_present cluster1 mirror test snap3 2021-02-20T08:09:48.756 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:09:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:09:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:09:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap3 2021-02-20T08:09:48.758 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:09:48.758 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:09:49.757 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 info mirror/test@snap3 2021-02-20T08:09:49.824 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:rbd image 'test': 2021-02-20T08:09:49.824 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: size 512 MiB in 128 objects 2021-02-20T08:09:49.824 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: order 22 (4 MiB objects) 2021-02-20T08:09:49.825 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: snapshot_count: 6 2021-02-20T08:09:49.825 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: id: 10a0aa806784 2021-02-20T08:09:49.825 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: block_name_prefix: rbd_data.10a0aa806784 2021-02-20T08:09:49.825 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: format: 2 2021-02-20T08:09:49.826 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: features: layering, exclusive-lock, object-map, fast-diff, non-primary 2021-02-20T08:09:49.826 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: op_features: 2021-02-20T08:09:49.826 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: flags: 2021-02-20T08:09:49.826 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: create_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:09:49.826 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: access_timestamp: Sat Feb 20 08:08:53 2021 2021-02-20T08:09:49.827 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: modify_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:09:49.827 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: protected: False 2021-02-20T08:09:49.827 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring state: enabled 2021-02-20T08:09:49.827 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring mode: snapshot 2021-02-20T08:09:49.828 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring global id: 3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd 2021-02-20T08:09:49.828 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring primary: false 2021-02-20T08:09:49.830 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:09:49.831 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snap2-clean ] 2021-02-20T08:09:49.832 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap2-clean 2021-02-20T08:09:49.832 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:09:49.832 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:09:49.833 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap2-clean 2021-02-20T08:09:49.833 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:09:49.833 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:09:49.834 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:09:49.834 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:09:49.835 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2-clean /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:09:50.490 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:09:50.494 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2-clean /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:09:51.288 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:09:51.291 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:09:51.566 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:09:51.692 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:09:51.693 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap3 2021-02-20T08:09:51.693 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:09:51.693 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:09:51.694 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap3 2021-02-20T08:09:51.694 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:09:51.694 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:09:51.694 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:09:51.695 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:09:51.695 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3 /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:09:52.374 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:09:52.379 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3 /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:09:53.243 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:09:53.246 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:09:53.524 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:09:53.653 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:09:53.653 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ clean_snap_name=snap3-clean 2021-02-20T08:09:53.653 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_snap cluster2 mirror test snap3-clean 2021-02-20T08:09:53.654 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:09:53.654 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:09:53.654 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:09:53.654 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap3-clean 2021-02-20T08:09:53.655 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap3-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T08:09:54.191 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T08:09:54.191 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:09:54.191 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:09:54.192 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:09:54.192 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:09:54.193 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=8 2021-02-20T08:09:54.193 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:09:54.193 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T08:09:54.760 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 34882560 bytes 2021-02-20T08:09:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 56763392 bytes 2021-02-20T08:09:55.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 73423872 bytes 2021-02-20T08:09:55.406 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 95011840 bytes 2021-02-20T08:09:55.568 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 121037824 bytes 2021-02-20T08:09:55.758 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 155942912 bytes 2021-02-20T08:09:55.998 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 228119552 bytes 2021-02-20T08:09:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 305959936 bytes 2021-02-20T08:09:56.441 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 317146112 bytes 2021-02-20T08:09:56.655 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 362880000 bytes 2021-02-20T08:09:56.907 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 457411584 bytes 2021-02-20T08:09:57.169 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 503277568 bytes 2021-02-20T08:09:57.608 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 560279552 bytes 2021-02-20T08:09:57.788 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 592822272 bytes 2021-02-20T08:09:58.027 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 653809664 bytes 2021-02-20T08:09:58.150 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 659851264 bytes 2021-02-20T08:09:58.388 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 719446016 bytes 2021-02-20T08:09:58.515 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 725410816 bytes 2021-02-20T08:09:58.671 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 750390272 bytes 2021-02-20T08:09:58.837 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 797941760 bytes 2021-02-20T08:09:59.076 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 821227520 bytes 2021-02-20T08:09:59.266 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 853140480 bytes 2021-02-20T08:09:59.504 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 911865856 bytes 2021-02-20T08:09:59.657 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 930348032 bytes 2021-02-20T08:09:59.806 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 947848192 bytes 2021-02-20T08:09:59.986 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 978600960 bytes 2021-02-20T08:10:00.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2700 IOs, 1043560448 bytes 2021-02-20T08:10:00.517 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2800 IOs, 1121285120 bytes 2021-02-20T08:10:00.752 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2900 IOs, 1187159040 bytes 2021-02-20T08:10:00.979 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3000 IOs, 1240509440 bytes 2021-02-20T08:10:01.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3100 IOs, 1270405120 bytes 2021-02-20T08:10:01.351 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3200 IOs, 1313278976 bytes 2021-02-20T08:10:01.478 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3300 IOs, 1325015040 bytes 2021-02-20T08:10:01.674 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3400 IOs, 1359746048 bytes 2021-02-20T08:10:01.857 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3500 IOs, 1394864128 bytes 2021-02-20T08:10:02.023 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3600 IOs, 1414392832 bytes 2021-02-20T08:10:02.183 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3700 IOs, 1431140352 bytes 2021-02-20T08:10:02.196 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:10:02.197 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:10:02.197 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:10:02.197 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:10:02.198 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:10:02.198 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:10:02.198 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:02.198 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:02.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:10:02.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local description_pattern= 2021-02-20T08:10:02.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local service_pattern= 2021-02-20T08:10:02.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:10:03.198 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:10:03.198 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:10:03.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:03.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:03.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:10:03.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local description_pattern= 2021-02-20T08:10:03.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local service_pattern= 2021-02-20T08:10:03.200 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T08:10:03.200 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T08:10:03.200 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:10:03.201 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:10:03.201 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T08:10:03.201 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ tee /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:10:03.274 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:test: 2021-02-20T08:10:03.275 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: global_id: 3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd 2021-02-20T08:10:03.275 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state: up+replaying 2021-02-20T08:10:03.275 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":524506112.0,"local_snapshot_timestamp":1613808525,"remote_snapshot_timestamp":1613808525,"replay_state":"idle"} 2021-02-20T08:10:03.276 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: service: mirror.0 on smithi090 2021-02-20T08:10:03.276 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: last_update: 2021-02-20 08:09:33 2021-02-20T08:10:03.276 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: peer_sites: 2021-02-20T08:10:03.276 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: name: cluster2 2021-02-20T08:10:03.276 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state: up+stopped 2021-02-20T08:10:03.277 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: description: local image is primary 2021-02-20T08:10:03.277 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: last_update: 2021-02-20 08:09:34 2021-02-20T08:10:03.288 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:10:03.289 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: state: up+replaying 2021-02-20T08:10:03.289 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^ description: .* /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:10:03.291 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":524506112.0,"local_snapshot_timestamp":1613808525,"remote_snapshot_timestamp":1613808525,"replay_state":"idle"} 2021-02-20T08:10:03.292 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:up+replaying 2021-02-20T08:10:03.292 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:10:03.293 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo up+replaying 2021-02-20T08:10:03.293 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^up+ 2021-02-20T08:10:03.294 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep service: *mirror..* on /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:10:03.295 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: service: mirror.0 on smithi090 2021-02-20T08:10:03.296 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:10:03.296 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local last_update=2021-02-20 08:09:33 2021-02-20T08:10:03.297 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 08:09:33 2021-02-20T08:10:03.297 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:10:03.297 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:03.298 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:03.298 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:10:03.299 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local prev_last_update=2021-02-20 08:09:33 2021-02-20T08:10:03.299 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T08:10:03.300 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T08:10:03.300 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:10:03.300 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror.mirror_status 2021-02-20T08:10:03.301 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T08:10:03.336 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T08:10:03.390 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local last_update state 2021-02-20T08:10:03.390 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T08:10:03.396 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ last_update=2021-02-20 08:09:33 2021-02-20T08:10:03.396 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T08:10:03.403 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ state=up+replaying 2021-02-20T08:10:03.403 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo up+replaying 2021-02-20T08:10:03.403 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep up+replaying 2021-02-20T08:10:03.404 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:up+replaying 2021-02-20T08:10:03.405 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:10:03.405 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:10:03.406 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap4 2021-02-20T08:10:03.406 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_snap cluster2 mirror test snap4 2021-02-20T08:10:03.406 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:10:03.406 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:03.407 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:03.407 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap4 2021-02-20T08:10:03.407 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap4 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:10:11.028 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7308572182473426) > weight (0.25) 2021-02-20T08:10:11.028 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.979122791669308) > weight (0.25) 2021-02-20T08:10:11.029 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.681755087905143) > weight (0.25) 2021-02-20T08:10:11.029 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6074860420779506) > weight (0.25) 2021-02-20T08:10:11.029 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T08:10:30.952 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T08:10:30.953 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:10:30.953 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:30.953 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:30.954 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T08:10:30.954 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:10:30.954 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:30.955 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:30.955 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state=started 2021-02-20T08:10:30.955 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:10:30.956 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:10:31.953 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T08:10:31.954 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:10:31.954 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:31.954 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:31.954 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local test_state=started 2021-02-20T08:10:31.955 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_result 2021-02-20T08:10:31.955 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_state=stopped 2021-02-20T08:10:31.956 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test+ 2021-02-20T08:10:31.956 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ greplocal -i cluster_instance=cluster1 2021-02-20T08:10:31.956 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state 2021-02-20T08:10:31.956 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:10:31.957 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:10:31.957 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance=cluster1 2021-02-20T08:10:31.957 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loop_instance 2021-02-20T08:10:31.957 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0 2021-02-20T08:10:31.957 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:10:31.958 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ seq 0 3 2021-02-20T08:10:31.959 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:10:31.960 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:0 2021-02-20T08:10:31.960 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:10:31.960 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:10:31.960 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:10:31.961 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:10:31.961 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:10:31.961 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:10:31.961 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:10:31.961 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:10:31.962 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:10:31.962 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:10:31.962 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:10:31.962 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:10:31.962 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:10:31.963 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:10:31.963 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:10:31.963 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:10:31.963 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:10:31.964 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:10:31.964 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:10:31.964 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:10:31.964 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:10:31.964 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:10:31.965 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:10:31.965 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:10:31.965 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:10:31.965 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:10:31.965 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:10:31.966 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:10:31.966 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:10:31.979 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:10:31.979 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:10:31.979 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:10:31.980 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:10:32.094 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:10:32.095 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ status_result= "state": "Replaying" 2021-02-20T08:10:32.096 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo "state": "Replaying" 2021-02-20T08:10:32.096 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep -i Replaying 2021-02-20T08:10:32.099 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: "state": "Replaying" 2021-02-20T08:10:32.099 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_state=started 2021-02-20T08:10:32.100 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test started = started 2021-02-20T08:10:32.100 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:10:32.100 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T08:10:32.101 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:10:32.101 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:10:32.101 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:32.102 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:32.102 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = journal ] 2021-02-20T08:10:32.102 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = snapshot ] 2021-02-20T08:10:32.103 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T08:10:32.103 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:10:32.104 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:10:32.104 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:32.104 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:32.105 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T08:10:32.105 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster2-mirror-test.status 2021-02-20T08:10:32.106 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:10:32.108 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:10:32.108 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T08:10:32.109 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror-test.status 2021-02-20T08:10:32.109 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:10:32.110 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:10:32.111 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T08:10:32.111 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:10:32.111 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:32.112 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:32.112 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T08:10:32.959 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Snapshot ID: 18 2021-02-20T08:10:32.980 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:10:32.981 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:10:32.981 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:32.982 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:32.982 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:10:32.982 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T08:10:32.982 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:10:33.037 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T08:10:33.040 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snapshot_id=18 2021-02-20T08:10:33.040 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:10:33.040 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.2 2021-02-20T08:10:33.241 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:10:33.241 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:10:33.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:33.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:33.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:10:33.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:10:33.243 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:10:33.312 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:10:33.317 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=14 2021-02-20T08:10:33.317 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 18 = 14 2021-02-20T08:10:33.317 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.4 2021-02-20T08:10:33.718 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:10:33.719 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:10:33.719 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:33.719 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:33.719 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:10:33.720 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:10:33.720 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:10:33.790 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:10:33.794 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=14 2021-02-20T08:10:33.795 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 18 = 14 2021-02-20T08:10:33.795 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.8 2021-02-20T08:10:34.596 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:10:34.597 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:10:34.597 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:34.597 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:34.597 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:10:34.598 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:10:34.598 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:10:34.669 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:10:34.670 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:10:34.674 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:10:34.674 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:10:34.674 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:10:34.675 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:10:34.675 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 18 = 2021-02-20T08:10:34.675 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1.6 2021-02-20T08:10:36.276 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:10:36.277 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:10:36.277 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:36.277 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:36.277 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:10:36.278 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:10:36.278 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:10:36.358 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:10:36.358 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:10:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:10:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:10:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:10:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:10:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 18 = 2021-02-20T08:10:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 2 2021-02-20T08:10:38.364 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:10:38.365 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:10:38.365 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:38.365 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:38.365 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:10:38.366 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:10:38.366 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:10:38.443 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:10:38.449 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=18 2021-02-20T08:10:38.449 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 18 = 18 2021-02-20T08:10:38.449 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:10:38.450 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snap_present cluster1 mirror test snap4 2021-02-20T08:10:38.450 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:10:38.450 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:38.450 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:38.451 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap4 2021-02-20T08:10:38.451 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:10:38.451 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:10:39.450 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 info mirror/test@snap4 2021-02-20T08:10:39.517 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:rbd image 'test': 2021-02-20T08:10:39.517 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: size 512 MiB in 128 objects 2021-02-20T08:10:39.518 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: order 22 (4 MiB objects) 2021-02-20T08:10:39.518 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: snapshot_count: 8 2021-02-20T08:10:39.518 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: id: 10a0aa806784 2021-02-20T08:10:39.518 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: block_name_prefix: rbd_data.10a0aa806784 2021-02-20T08:10:39.519 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: format: 2 2021-02-20T08:10:39.519 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: features: layering, exclusive-lock, object-map, fast-diff, non-primary 2021-02-20T08:10:39.519 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: op_features: 2021-02-20T08:10:39.519 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: flags: 2021-02-20T08:10:39.520 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: create_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:10:39.520 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: access_timestamp: Sat Feb 20 08:08:53 2021 2021-02-20T08:10:39.520 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: modify_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:10:39.520 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: protected: False 2021-02-20T08:10:39.521 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring state: enabled 2021-02-20T08:10:39.521 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring mode: snapshot 2021-02-20T08:10:39.521 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring global id: 3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd 2021-02-20T08:10:39.521 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring primary: false 2021-02-20T08:10:39.522 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:10:39.523 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snap3-clean ] 2021-02-20T08:10:39.523 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap3-clean 2021-02-20T08:10:39.523 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:39.523 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:39.524 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap3-clean 2021-02-20T08:10:39.524 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:10:39.524 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:10:39.524 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:10:39.525 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:10:39.525 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3-clean /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:10:40.159 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:10:40.163 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3-clean /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:10:41.000 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:10:41.004 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:10:41.275 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:10:41.403 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:10:41.403 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap4 2021-02-20T08:10:41.403 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:41.403 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:41.404 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap4 2021-02-20T08:10:41.404 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:10:41.404 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:10:41.404 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:10:41.404 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:10:41.405 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4 /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:10:42.062 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:10:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4 /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:10:42.860 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:10:42.863 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:10:43.144 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:10:43.272 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:10:43.273 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ clean_snap_name=snap4-clean 2021-02-20T08:10:43.273 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_snap cluster2 mirror test snap4-clean 2021-02-20T08:10:43.273 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:10:43.274 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:43.274 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:43.274 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap4-clean 2021-02-20T08:10:43.274 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap4-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T08:10:44.004 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T08:10:44.004 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:10:44.004 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:44.005 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:44.005 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:10:44.006 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=9 2021-02-20T08:10:44.006 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:10:44.006 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T08:10:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 57204736 bytes 2021-02-20T08:10:45.051 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 107130880 bytes 2021-02-20T08:10:45.235 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 114525184 bytes 2021-02-20T08:10:45.461 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 159052800 bytes 2021-02-20T08:10:45.604 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 171954176 bytes 2021-02-20T08:10:45.738 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 196956160 bytes 2021-02-20T08:10:46.018 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 269254656 bytes 2021-02-20T08:10:46.231 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 293164032 bytes 2021-02-20T08:10:46.383 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 311298048 bytes 2021-02-20T08:10:46.539 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 327937024 bytes 2021-02-20T08:10:46.815 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 399113216 bytes 2021-02-20T08:10:47.063 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 467594240 bytes 2021-02-20T08:10:47.354 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 546270208 bytes 2021-02-20T08:10:47.468 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 546884608 bytes 2021-02-20T08:10:47.647 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 577579008 bytes 2021-02-20T08:10:47.857 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 616970240 bytes 2021-02-20T08:10:48.068 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 662163456 bytes 2021-02-20T08:10:48.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 686214144 bytes 2021-02-20T08:10:48.533 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 762738688 bytes 2021-02-20T08:10:48.754 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 809655296 bytes 2021-02-20T08:10:48.940 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 828727296 bytes 2021-02-20T08:10:49.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 904633344 bytes 2021-02-20T08:10:49.496 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 963575808 bytes 2021-02-20T08:10:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 976913408 bytes 2021-02-20T08:10:49.778 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 984537088 bytes 2021-02-20T08:10:50.070 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 1074797568 bytes 2021-02-20T08:10:50.298 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2700 IOs, 1115512832 bytes 2021-02-20T08:10:50.520 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2800 IOs, 1166370816 bytes 2021-02-20T08:10:50.706 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2900 IOs, 1201555456 bytes 2021-02-20T08:10:50.938 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3000 IOs, 1253936128 bytes 2021-02-20T08:10:51.128 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3100 IOs, 1286065152 bytes 2021-02-20T08:10:51.345 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3200 IOs, 1332346880 bytes 2021-02-20T08:10:51.513 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3300 IOs, 1350420480 bytes 2021-02-20T08:10:51.962 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3400 IOs, 1399113728 bytes 2021-02-20T08:10:52.188 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3500 IOs, 1432574976 bytes 2021-02-20T08:10:52.393 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3600 IOs, 1468555264 bytes 2021-02-20T08:10:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3700 IOs, 1479731200 bytes 2021-02-20T08:10:52.741 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3800 IOs, 1514277888 bytes 2021-02-20T08:10:53.010 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:10:53.011 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:10:53.011 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:10:53.011 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:10:53.011 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:10:53.012 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:10:53.012 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:53.012 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:53.012 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:10:53.012 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local description_pattern= 2021-02-20T08:10:53.013 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local service_pattern= 2021-02-20T08:10:53.013 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:10:54.012 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:10:54.012 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:10:54.012 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:54.013 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:54.013 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:10:54.013 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local description_pattern= 2021-02-20T08:10:54.014 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local service_pattern= 2021-02-20T08:10:54.014 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T08:10:54.015 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T08:10:54.015 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:10:54.016 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:10:54.017 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T08:10:54.017 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ tee /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:10:54.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:test: 2021-02-20T08:10:54.090 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: global_id: 3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd 2021-02-20T08:10:54.090 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state: up+replaying 2021-02-20T08:10:54.091 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":531720192.0,"local_snapshot_timestamp":1613808583,"remote_snapshot_timestamp":1613808583,"replay_state":"idle"} 2021-02-20T08:10:54.091 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: service: mirror.0 on smithi090 2021-02-20T08:10:54.092 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: last_update: 2021-02-20 08:10:33 2021-02-20T08:10:54.092 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: peer_sites: 2021-02-20T08:10:54.093 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: name: cluster2 2021-02-20T08:10:54.093 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state: up+stopped 2021-02-20T08:10:54.093 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: description: local image is primary 2021-02-20T08:10:54.094 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: last_update: 2021-02-20 08:10:34 2021-02-20T08:10:54.102 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:10:54.103 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: state: up+replaying 2021-02-20T08:10:54.104 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^ description: .* /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:10:54.106 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":531720192.0,"local_snapshot_timestamp":1613808583,"remote_snapshot_timestamp":1613808583,"replay_state":"idle"} 2021-02-20T08:10:54.106 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:10:54.106 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo up+replaying 2021-02-20T08:10:54.107 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^up+ 2021-02-20T08:10:54.107 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:up+replaying 2021-02-20T08:10:54.107 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep service: *mirror..* on /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:10:54.109 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: service: mirror.0 on smithi090 2021-02-20T08:10:54.109 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:10:54.112 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local last_update=2021-02-20 08:10:33 2021-02-20T08:10:54.112 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 08:10:33 2021-02-20T08:10:54.112 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:10:54.112 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:54.112 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:54.113 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:10:54.113 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local prev_last_update=2021-02-20 08:10:33 2021-02-20T08:10:54.113 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T08:10:54.114 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T08:10:54.115 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:10:54.115 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror.mirror_status 2021-02-20T08:10:54.116 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T08:10:54.156 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T08:10:54.209 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local last_update state 2021-02-20T08:10:54.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T08:10:54.215 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ last_update=2021-02-20 08:10:33 2021-02-20T08:10:54.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T08:10:54.221 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ state=up+replaying 2021-02-20T08:10:54.221 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo up+replaying 2021-02-20T08:10:54.221 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep up+replaying 2021-02-20T08:10:54.223 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:up+replaying 2021-02-20T08:10:54.223 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:10:54.223 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:10:54.223 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap5 2021-02-20T08:10:54.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_snap cluster2 mirror test snap5 2021-02-20T08:10:54.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:10:54.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:10:54.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:10:54.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap5 2021-02-20T08:10:54.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap5 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:11:20.030 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.509118519373825) > weight (0.25) 2021-02-20T08:11:20.031 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9504660946115948) > weight (0.25) 2021-02-20T08:11:20.031 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T08:11:20.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T08:11:20.032 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T08:11:20.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T08:11:20.036+0000 7fc4bcd4b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 18979) UID: 1000 2021-02-20T08:11:20.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:11:20.965 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T08:11:20.966 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:11:20.966 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:11:20.967 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:11:20.967 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T08:11:20.968 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:11:20.968 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:11:20.968 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:11:20.969 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state=started 2021-02-20T08:11:20.969 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:11:20.970 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:11:21.967 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T08:11:21.967 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:11:21.967 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:11:21.968 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:11:21.968 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local test_state=started 2021-02-20T08:11:21.968 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_result 2021-02-20T08:11:21.969 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_state=stopped 2021-02-20T08:11:21.969 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T08:11:21.970 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_instance=cluster1 2021-02-20T08:11:21.970 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:11:21.970 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:11:21.971 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ + local instance=cluster1 2021-02-20T08:11:21.971 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loop_instance 2021-02-20T08:11:21.972 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:grep+ -isleep state 0 2021-02-20T08:11:21.972 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:11:21.977 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:11:21.977 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ seq 0 3 2021-02-20T08:11:21.978 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:11:21.978 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:0 2021-02-20T08:11:21.979 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:11:21.979 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:11:21.979 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:11:21.979 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:11:21.980 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:11:21.980 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:11:21.980 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:11:21.980 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:11:21.980 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:11:21.981 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:11:21.981 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:11:21.981 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:11:21.981 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:11:21.981 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:11:21.982 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:11:21.982 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:11:21.982 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:11:21.982 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:11:21.983 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:11:21.983 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:11:21.983 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:11:21.983 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:11:21.983 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:11:21.984 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:11:21.984 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:11:21.984 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:11:21.984 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:11:21.985 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:11:21.985 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:11:21.993 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:11:21.994 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:11:21.994 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:11:21.994 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:11:22.112 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:11:22.113 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ status_result= "state": "Replaying" 2021-02-20T08:11:22.113 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo "state": "Replaying" 2021-02-20T08:11:22.114 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep -i Replaying 2021-02-20T08:11:22.119 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: "state": "Replaying" 2021-02-20T08:11:22.120 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_state=started 2021-02-20T08:11:22.120 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test started = started 2021-02-20T08:11:22.121 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:11:22.121 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T08:11:22.121 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:11:22.122 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:11:22.122 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:11:22.123 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:11:22.123 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = journal ] 2021-02-20T08:11:22.124 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = snapshot ] 2021-02-20T08:11:22.124 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T08:11:22.124 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:11:22.125 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:11:22.125 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:11:22.126 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:11:22.126 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T08:11:22.127 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster2-mirror-test.status 2021-02-20T08:11:22.130 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:11:22.131 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:11:22.131 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T08:11:22.131 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror-test.status 2021-02-20T08:11:22.132 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:11:22.132 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:11:22.132 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T08:11:22.132 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:11:22.133 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:11:22.133 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:11:22.133 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T08:11:22.972 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Snapshot ID: 22 2021-02-20T08:11:22.995 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:11:22.995 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:11:22.995 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:11:22.996 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:11:22.996 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:11:22.996 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T08:11:22.996 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:11:23.034 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T08:11:23.034 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:11:23.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:11:23.035 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T08:11:23.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T08:11:23.036 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T08:11:23.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T08:11:23.041 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T08:11:23.061 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T08:11:23.065 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snapshot_id=22 2021-02-20T08:11:23.065 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:11:23.065 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.2 2021-02-20T08:11:23.128 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6348436259844051) > weight (0.25) 2021-02-20T08:11:23.128 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.40128554768685176) > weight (0.25) 2021-02-20T08:11:23.129 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5410825760975475) > weight (0.25) 2021-02-20T08:11:23.129 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9498863281676599) > weight (0.25) 2021-02-20T08:11:23.129 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T08:11:23.266 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:11:23.266 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:11:23.266 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:11:23.267 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:11:23.267 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:11:23.267 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ + xmlstarlet sel -t -crbd //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:11:23.267 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:11:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:11:23.329 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=18 2021-02-20T08:11:23.329 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 22 = 18 2021-02-20T08:11:23.330 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.4 2021-02-20T08:11:23.731 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:11:23.731 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:11:23.732 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:11:23.732 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:11:23.732 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:11:23.732 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:11:23.733 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:11:23.799 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:11:23.804 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=18 2021-02-20T08:11:23.805 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 22 = 18 2021-02-20T08:11:23.805 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.8 2021-02-20T08:11:24.607 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:11:24.607 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:11:24.607 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:11:24.607 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:11:24.608 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:11:24.608 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:11:24.608 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:11:24.675 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:11:24.675 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:11:24.679 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:11:24.680 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:11:24.680 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:11:24.681 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:11:24.681 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 22 = 2021-02-20T08:11:24.681 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1.6 2021-02-20T08:11:26.282 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:11:26.282 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:11:26.282 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:11:26.283 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:11:26.283 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:11:26.283 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:11:26.283 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:11:26.362 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:11:26.363 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:11:26.368 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:11:26.368 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:11:26.369 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:11:26.369 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:11:26.370 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 22 = 2021-02-20T08:11:26.370 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 2 2021-02-20T08:11:28.369 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:11:28.369 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:11:28.369 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:11:28.370 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:11:28.370 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:11:28.370 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:11:28.371 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:11:28.442 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:11:28.448 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=22 2021-02-20T08:11:28.449 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 22 = 22 2021-02-20T08:11:28.449 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:11:28.449 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snap_present cluster1 mirror test snap5 2021-02-20T08:11:28.449 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:11:28.450 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:11:28.450 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:11:28.450 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap5 2021-02-20T08:11:28.450 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:11:28.451 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:11:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 info mirror/test@snap5 2021-02-20T08:11:29.519 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:rbd image 'test': 2021-02-20T08:11:29.520 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: size 512 MiB in 128 objects 2021-02-20T08:11:29.520 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: order 22 (4 MiB objects) 2021-02-20T08:11:29.520 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: snapshot_count: 10 2021-02-20T08:11:29.521 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: id: 10a0aa806784 2021-02-20T08:11:29.521 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: block_name_prefix: rbd_data.10a0aa806784 2021-02-20T08:11:29.521 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: format: 2 2021-02-20T08:11:29.521 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: features: layering, exclusive-lock, object-map, fast-diff, non-primary 2021-02-20T08:11:29.521 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: op_features: 2021-02-20T08:11:29.522 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: flags: 2021-02-20T08:11:29.522 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: create_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:11:29.522 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: access_timestamp: Sat Feb 20 08:10:40 2021 2021-02-20T08:11:29.522 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: modify_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:11:29.522 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: protected: False 2021-02-20T08:11:29.523 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring state: enabled 2021-02-20T08:11:29.523 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring mode: snapshot 2021-02-20T08:11:29.523 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring global id: 3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd 2021-02-20T08:11:29.523 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring primary: false 2021-02-20T08:11:29.526 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:11:29.526 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snap4-clean ] 2021-02-20T08:11:29.526 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap4-clean 2021-02-20T08:11:29.527 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:11:29.527 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:11:29.527 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap4-clean 2021-02-20T08:11:29.527 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:11:29.528 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:11:29.528 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:11:29.528 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:11:29.528 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4-clean /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:11:30.167 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:11:30.171 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4-clean /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:11:31.028 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:11:31.031 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:11:31.306 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:11:31.430 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:11:31.431 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap5 2021-02-20T08:11:31.431 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:11:31.431 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:11:31.431 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap5 2021-02-20T08:11:31.432 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:11:31.432 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:11:31.432 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:11:31.432 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:11:31.433 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5 /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:11:32.106 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:11:32.111 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5 /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:11:32.926 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:11:32.930 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:11:33.208 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:11:33.497 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:11:33.498 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ clean_snap_name=snap5-clean 2021-02-20T08:11:33.498 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_snap cluster2 mirror test snap5-clean 2021-02-20T08:11:33.499 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:11:33.499 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:11:33.499 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:11:33.500 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap5-clean 2021-02-20T08:11:33.500 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap5-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T08:11:34.022 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T08:11:34.023 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:11:34.024 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:11:34.024 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:11:34.025 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:11:34.027 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=6 2021-02-20T08:11:34.028 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:11:34.028 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T08:11:34.709 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 26863616 bytes 2021-02-20T08:11:34.994 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 41338880 bytes 2021-02-20T08:11:35.292 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 114020352 bytes 2021-02-20T08:11:35.438 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 114622464 bytes 2021-02-20T08:11:35.679 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 165735424 bytes 2021-02-20T08:11:35.942 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 232557568 bytes 2021-02-20T08:11:36.223 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 313407488 bytes 2021-02-20T08:11:36.508 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 397660160 bytes 2021-02-20T08:11:36.702 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 415798272 bytes 2021-02-20T08:11:36.883 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 443812864 bytes 2021-02-20T08:11:37.187 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 537946112 bytes 2021-02-20T08:11:37.453 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 584115200 bytes 2021-02-20T08:11:37.632 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 608491520 bytes 2021-02-20T08:11:37.953 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 694751232 bytes 2021-02-20T08:11:38.233 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 764796928 bytes 2021-02-20T08:11:38.390 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 776375296 bytes 2021-02-20T08:11:38.576 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 804776960 bytes 2021-02-20T08:11:38.740 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 821147648 bytes 2021-02-20T08:11:39.045 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 894664704 bytes 2021-02-20T08:11:39.355 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 978663424 bytes 2021-02-20T08:11:39.552 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 1012024320 bytes 2021-02-20T08:11:39.783 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 1063998464 bytes 2021-02-20T08:11:39.961 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 1084791808 bytes 2021-02-20T08:11:40.025 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:11:40.025 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:11:40.025 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:11:40.026 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:11:40.026 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:11:40.026 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:11:40.026 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:11:40.027 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:11:40.027 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:11:40.027 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local description_pattern= 2021-02-20T08:11:40.027 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local service_pattern= 2021-02-20T08:11:40.028 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:11:41.026 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:11:41.027 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:11:41.027 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:11:41.027 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:11:41.027 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:11:41.027 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local description_pattern= 2021-02-20T08:11:41.028 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local service_pattern= 2021-02-20T08:11:41.028 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T08:11:41.028 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T08:11:41.028 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:11:41.029 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:11:41.029 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T08:11:41.030 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ tee /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:11:41.107 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:test: 2021-02-20T08:11:41.108 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: global_id: 3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd 2021-02-20T08:11:41.108 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state: up+replaying 2021-02-20T08:11:41.108 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: description: replaying, {"bytes_per_second":16787012.27,"bytes_per_snapshot":489072128.0,"local_snapshot_timestamp":1613808682,"remote_snapshot_timestamp":1613808682,"replay_state":"idle"} 2021-02-20T08:11:41.108 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: service: mirror.0 on smithi090 2021-02-20T08:11:41.109 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: last_update: 2021-02-20 08:11:33 2021-02-20T08:11:41.109 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: peer_sites: 2021-02-20T08:11:41.109 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: name: cluster2 2021-02-20T08:11:41.109 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state: up+stopped 2021-02-20T08:11:41.109 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: description: local image is primary 2021-02-20T08:11:41.110 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: last_update: 2021-02-20 08:11:34 2021-02-20T08:11:41.118 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:11:41.119 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: state: up+replaying 2021-02-20T08:11:41.120 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^ description: .* /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:11:41.122 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: description: replaying, {"bytes_per_second":16787012.27,"bytes_per_snapshot":489072128.0,"local_snapshot_timestamp":1613808682,"remote_snapshot_timestamp":1613808682,"replay_state":"idle"} 2021-02-20T08:11:41.122 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:up+replaying 2021-02-20T08:11:41.123 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:11:41.123 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo up+replaying 2021-02-20T08:11:41.123 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^up+ 2021-02-20T08:11:41.124 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep service: *mirror..* on /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:11:41.124 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: service: mirror.0 on smithi090 2021-02-20T08:11:41.125 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:11:41.127 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local last_update=2021-02-20 08:11:33 2021-02-20T08:11:41.127 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 08:11:33 2021-02-20T08:11:41.128 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:11:41.128 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:11:41.128 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:11:41.129 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:11:41.129 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local prev_last_update=2021-02-20 08:11:33 2021-02-20T08:11:41.130 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T08:11:41.130 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T08:11:41.130 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:11:41.132 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror.mirror_status 2021-02-20T08:11:41.132 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T08:11:41.173 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T08:11:41.229 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local last_update state 2021-02-20T08:11:41.230 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T08:11:41.235 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ last_update=2021-02-20 08:11:33 2021-02-20T08:11:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T08:11:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ state=up+replaying 2021-02-20T08:11:41.241 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo up+replaying 2021-02-20T08:11:41.241 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep up+replaying 2021-02-20T08:11:41.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:up+replaying 2021-02-20T08:11:41.243 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:11:41.243 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:11:41.243 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap6 2021-02-20T08:11:41.243 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_snap cluster2 mirror test snap6 2021-02-20T08:11:41.244 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:11:41.244 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:11:41.244 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:11:41.244 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap6 2021-02-20T08:11:41.245 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap6 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:12:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T08:12:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:12:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:12:11.003 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:12:11.003 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T08:12:11.003 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:12:11.003 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:12:11.004 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:12:11.004 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state=started 2021-02-20T08:12:11.004 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:12:11.004 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:12:12.003 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T08:12:12.003 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:12:12.003 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:12:12.003 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:12:12.004 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local test_state=started 2021-02-20T08:12:12.004 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_result 2021-02-20T08:12:12.004 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_state=stopped 2021-02-20T08:12:12.005 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T08:12:12.005 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ + local cluster_instance=cluster1 2021-02-20T08:12:12.006 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:12:12.006 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:grep -i state 2021-02-20T08:12:12.006 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:12:12.006 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance=cluster1 2021-02-20T08:12:12.006 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loop_instance 2021-02-20T08:12:12.007 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0 2021-02-20T08:12:12.007 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:12:12.007 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ seq 0 3 2021-02-20T08:12:12.009 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:12:12.009 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:0 2021-02-20T08:12:12.009 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:12:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:12:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:12:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:12:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:12:12.011 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:12:12.011 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:12:12.011 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:12:12.011 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:12:12.011 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:12:12.012 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:12:12.012 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:12:12.012 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:12:12.012 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:12:12.012 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:12:12.013 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:12:12.013 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:12:12.013 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:12:12.013 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:12:12.014 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:12:12.014 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:12:12.014 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:12:12.014 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:12:12.014 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:12:12.015 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:12:12.015 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:12:12.015 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:12:12.015 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:12:12.016 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:12:12.031 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:12:12.031 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:12:12.032 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:12:12.032 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:12:12.149 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:12:12.150 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ status_result= "state": "Replaying" 2021-02-20T08:12:12.150 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo "state": "Replaying" 2021-02-20T08:12:12.150 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep -i Replaying 2021-02-20T08:12:12.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: "state": "Replaying" 2021-02-20T08:12:12.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_state=started 2021-02-20T08:12:12.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test started = started 2021-02-20T08:12:12.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:12:12.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T08:12:12.154 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:12:12.154 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:12:12.154 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:12:12.154 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:12:12.155 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = journal ] 2021-02-20T08:12:12.155 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = snapshot ] 2021-02-20T08:12:12.155 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T08:12:12.155 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:12:12.155 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:12:12.156 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:12:12.156 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:12:12.156 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T08:12:12.156 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster2-mirror-test.status 2021-02-20T08:12:12.156 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:12:12.158 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:12:12.158 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T08:12:12.158 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror-test.status 2021-02-20T08:12:12.158 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:12:12.159 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:12:12.159 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T08:12:12.159 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:12:12.159 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:12:12.160 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:12:12.160 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T08:12:13.026 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Snapshot ID: 26 2021-02-20T08:12:13.050 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:12:13.050 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:12:13.051 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:12:13.051 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:12:13.052 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:12:13.052 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T08:12:13.053 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:12:13.114 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T08:12:13.118 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snapshot_id=26 2021-02-20T08:12:13.118 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:12:13.118 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.2 2021-02-20T08:12:13.319 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:12:13.320 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:12:13.320 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:12:13.320 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:12:13.320 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:12:13.321 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:12:13.321 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:12:13.600 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:12:13.605 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=22 2021-02-20T08:12:13.605 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 26 = 22 2021-02-20T08:12:13.606 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.4 2021-02-20T08:12:14.007 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:12:14.007 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:12:14.008 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:12:14.008 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:12:14.008 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:12:14.008 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:12:14.009 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:12:14.087 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:12:14.088 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:12:14.092 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:12:14.093 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:12:14.093 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:12:14.094 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:12:14.094 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 26 = 2021-02-20T08:12:14.094 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.8 2021-02-20T08:12:14.895 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:12:14.896 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:12:14.896 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:12:14.896 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:12:14.897 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:12:14.897 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:12:14.897 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:12:14.967 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:12:14.968 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:12:14.972 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:12:14.972 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:12:14.972 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:12:14.973 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:12:14.973 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 26 = 2021-02-20T08:12:14.973 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1.6 2021-02-20T08:12:16.575 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:12:16.575 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:12:16.575 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:12:16.576 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:12:16.576 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:12:16.576 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:12:16.576 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:12:16.645 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:12:16.646 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:12:16.650 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:12:16.651 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:12:16.651 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:12:16.651 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:12:16.651 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 26 = 2021-02-20T08:12:16.652 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 2 2021-02-20T08:12:18.653 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:12:18.653 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:12:18.653 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:12:18.654 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:12:18.654 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:12:18.654 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:12:18.654 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:12:18.749 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:12:18.756 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=26 2021-02-20T08:12:18.756 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 26 = 26 2021-02-20T08:12:18.757 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:12:18.757 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snap_present cluster1 mirror test snap6 2021-02-20T08:12:18.757 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:12:18.757 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:12:18.758 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:12:18.758 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap6 2021-02-20T08:12:18.758 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:12:18.758 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:12:19.756 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 info mirror/test@snap6 2021-02-20T08:12:19.824 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:rbd image 'test': 2021-02-20T08:12:19.824 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: size 512 MiB in 128 objects 2021-02-20T08:12:19.825 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: order 22 (4 MiB objects) 2021-02-20T08:12:19.825 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: snapshot_count: 12 2021-02-20T08:12:19.825 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: id: 10a0aa806784 2021-02-20T08:12:19.825 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: block_name_prefix: rbd_data.10a0aa806784 2021-02-20T08:12:19.826 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: format: 2 2021-02-20T08:12:19.826 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: features: layering, exclusive-lock, object-map, fast-diff, non-primary 2021-02-20T08:12:19.826 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: op_features: 2021-02-20T08:12:19.826 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: flags: 2021-02-20T08:12:19.827 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: create_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:12:19.827 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: access_timestamp: Sat Feb 20 08:10:40 2021 2021-02-20T08:12:19.827 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: modify_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:12:19.827 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: protected: False 2021-02-20T08:12:19.827 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring state: enabled 2021-02-20T08:12:19.828 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring mode: snapshot 2021-02-20T08:12:19.828 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring global id: 3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd 2021-02-20T08:12:19.828 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring primary: false 2021-02-20T08:12:19.829 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:12:19.830 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snap5-clean ] 2021-02-20T08:12:19.830 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap5-clean 2021-02-20T08:12:19.830 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:12:19.831 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:12:19.831 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap5-clean 2021-02-20T08:12:19.831 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:12:19.831 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:12:19.831 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:12:19.832 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:12:19.832 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5-clean /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:12:20.452 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:12:20.456 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5-clean /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:12:21.470 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:12:21.474 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:12:21.749 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:12:21.874 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:12:21.875 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap6 2021-02-20T08:12:21.875 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:12:21.875 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:12:21.875 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap6 2021-02-20T08:12:21.876 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:12:21.876 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:12:21.876 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:12:21.876 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:12:21.877 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6 /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:12:22.497 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:12:22.502 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6 /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:12:23.263 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:12:23.266 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:12:23.544 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:12:23.671 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:12:23.671 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ clean_snap_name=snap6-clean 2021-02-20T08:12:23.672 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_snap cluster2 mirror test snap6-clean 2021-02-20T08:12:23.672 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:12:23.672 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:12:23.672 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:12:23.672 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap6-clean 2021-02-20T08:12:23.673 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap6-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T08:12:24.052 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T08:12:24.052 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:12:24.053 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:12:24.053 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:12:24.053 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:12:24.054 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=9 2021-02-20T08:12:24.054 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:12:24.054 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T08:12:24.691 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 9430016 bytes 2021-02-20T08:12:25.052 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 50434048 bytes 2021-02-20T08:12:25.348 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 91559936 bytes 2021-02-20T08:12:25.583 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 132841472 bytes 2021-02-20T08:12:25.806 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 178197504 bytes 2021-02-20T08:12:25.943 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 180319232 bytes 2021-02-20T08:12:26.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 282285056 bytes 2021-02-20T08:12:26.546 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 334227456 bytes 2021-02-20T08:12:26.771 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 379184128 bytes 2021-02-20T08:12:27.003 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 420402176 bytes 2021-02-20T08:12:27.217 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 456537088 bytes 2021-02-20T08:12:27.362 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 469489664 bytes 2021-02-20T08:12:27.502 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 475118592 bytes 2021-02-20T08:12:27.722 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 522673152 bytes 2021-02-20T08:12:27.891 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 553955328 bytes 2021-02-20T08:12:28.075 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 568580096 bytes 2021-02-20T08:12:28.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 592074752 bytes 2021-02-20T08:12:28.483 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 639880192 bytes 2021-02-20T08:12:28.732 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 701707264 bytes 2021-02-20T08:12:29.026 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 756163584 bytes 2021-02-20T08:12:29.244 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 802657280 bytes 2021-02-20T08:12:29.492 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 847711232 bytes 2021-02-20T08:12:29.800 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 921519104 bytes 2021-02-20T08:12:29.976 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 941190144 bytes 2021-02-20T08:12:30.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 984574976 bytes 2021-02-20T08:12:30.469 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 1047526400 bytes 2021-02-20T08:12:30.674 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2700 IOs, 1086417920 bytes 2021-02-20T08:12:30.873 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2800 IOs, 1120156672 bytes 2021-02-20T08:12:31.103 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2900 IOs, 1166609408 bytes 2021-02-20T08:12:31.289 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3000 IOs, 1196264448 bytes 2021-02-20T08:12:31.421 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3100 IOs, 1198134272 bytes 2021-02-20T08:12:31.590 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3200 IOs, 1229242368 bytes 2021-02-20T08:12:31.955 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3300 IOs, 1317252096 bytes 2021-02-20T08:12:32.157 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3400 IOs, 1352807424 bytes 2021-02-20T08:12:32.492 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3500 IOs, 1435166720 bytes 2021-02-20T08:12:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3600 IOs, 1480090624 bytes 2021-02-20T08:12:32.937 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3700 IOs, 1523441664 bytes 2021-02-20T08:12:33.059 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:12:33.059 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:12:33.059 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:12:33.059 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:12:33.060 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:12:33.060 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:12:33.060 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:12:33.060 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:12:33.061 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:12:33.061 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local description_pattern= 2021-02-20T08:12:33.061 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local service_pattern= 2021-02-20T08:12:33.061 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:12:34.060 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:12:34.060 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:12:34.061 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:12:34.061 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:12:34.061 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:12:34.061 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local description_pattern= 2021-02-20T08:12:34.061 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local service_pattern= 2021-02-20T08:12:34.062 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T08:12:34.062 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T08:12:34.063 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:12:34.063 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:12:34.063 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T08:12:34.064 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ tee /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:12:34.129 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:test: 2021-02-20T08:12:34.130 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: global_id: 3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd 2021-02-20T08:12:34.130 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state: up+replaying 2021-02-20T08:12:34.130 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: description: replaying, {"bytes_per_second":16624469.33,"bytes_per_snapshot":501172224.0,"local_snapshot_timestamp":1613808732,"remote_snapshot_timestamp":1613808732,"replay_state":"idle"} 2021-02-20T08:12:34.130 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: service: mirror.0 on smithi090 2021-02-20T08:12:34.131 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: last_update: 2021-02-20 08:12:33 2021-02-20T08:12:34.131 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: peer_sites: 2021-02-20T08:12:34.131 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: name: cluster2 2021-02-20T08:12:34.131 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state: up+stopped 2021-02-20T08:12:34.131 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: description: local image is primary 2021-02-20T08:12:34.132 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: last_update: 2021-02-20 08:12:04 2021-02-20T08:12:34.140 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:12:34.141 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: state: up+replaying 2021-02-20T08:12:34.146 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^ description: .* /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:12:34.149 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: description: replaying, {"bytes_per_second":16624469.33,"bytes_per_snapshot":501172224.0,"local_snapshot_timestamp":1613808732,"remote_snapshot_timestamp":1613808732,"replay_state":"idle"} 2021-02-20T08:12:34.150 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:up+replaying 2021-02-20T08:12:34.150 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: service: mirror.0 on smithi090 2021-02-20T08:12:34.150 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:12:34.151 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo up+replaying 2021-02-20T08:12:34.151 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^up+ 2021-02-20T08:12:34.151 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep service: *mirror..* on /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:12:34.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:12:34.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local last_update=2021-02-20 08:12:33 2021-02-20T08:12:34.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 08:12:33 2021-02-20T08:12:34.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:12:34.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:12:34.154 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:12:34.154 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:12:34.154 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local prev_last_update=2021-02-20 08:12:33 2021-02-20T08:12:34.154 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T08:12:34.154 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T08:12:34.155 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:12:34.155 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror.mirror_status 2021-02-20T08:12:34.155 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T08:12:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T08:12:34.235 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local last_update state 2021-02-20T08:12:34.236 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T08:12:34.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ last_update=2021-02-20 08:12:33 2021-02-20T08:12:34.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T08:12:34.244 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ state=up+replaying 2021-02-20T08:12:34.244 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo up+replaying 2021-02-20T08:12:34.245 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep up+replaying 2021-02-20T08:12:34.246 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:up+replaying 2021-02-20T08:12:34.246 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:12:34.247 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:12:34.247 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap7 2021-02-20T08:12:34.247 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_snap cluster2 mirror test snap7 2021-02-20T08:12:34.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:12:34.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:12:34.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:12:34.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap7 2021-02-20T08:12:34.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap7 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:12:48.042 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8934305039903901) > weight (0.25) 2021-02-20T08:12:48.043 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5422273566796454) > weight (0.25) 2021-02-20T08:12:48.043 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7332459845354252) > weight (0.25) 2021-02-20T08:12:48.043 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.49220303403042276) > weight (0.25) 2021-02-20T08:12:48.044 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T08:13:03.048 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T08:13:03.049 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:13:03.049 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:03.050 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:03.050 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T08:13:03.050 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:13:03.051 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:03.051 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:03.051 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state=started 2021-02-20T08:13:03.051 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:13:03.052 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:13:04.050 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T08:13:04.051 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:13:04.051 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:04.051 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:04.052 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local test_state=started 2021-02-20T08:13:04.052 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_result 2021-02-20T08:13:04.053 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_state=stopped 2021-02-20T08:13:04.053 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T08:13:04.053 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ + local cluster_instance=cluster1 2021-02-20T08:13:04.054 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:13:04.054 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ localgrep cluster=cluster1 2021-02-20T08:13:04.054 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: -i state 2021-02-20T08:13:04.055 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance=cluster1 2021-02-20T08:13:04.055 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loop_instance 2021-02-20T08:13:04.055 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0 2021-02-20T08:13:04.058 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:13:04.059 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ seq 0 3 2021-02-20T08:13:04.059 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:13:04.059 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:0 2021-02-20T08:13:04.060 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:13:04.060 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:13:04.060 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:13:04.060 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:13:04.060 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:13:04.061 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:13:04.061 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:13:04.061 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:13:04.061 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:13:04.062 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:13:04.062 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:13:04.062 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:13:04.062 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:13:04.063 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:13:04.063 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:13:04.063 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:13:04.063 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:13:04.063 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:13:04.064 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:13:04.064 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:13:04.064 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:13:04.064 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:13:04.064 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:13:04.065 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:13:04.065 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:13:04.065 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:13:04.065 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:13:04.065 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:13:04.066 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:13:04.076 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:13:04.076 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:13:04.076 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:13:04.077 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:13:04.131 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T08:13:04.132 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T08:13:04.132 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T08:13:04.133 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T08:13:04.129+0000 7fb2454e4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 21412) UID: 1000 2021-02-20T08:13:04.208 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:13:04.214 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ status_result= "state": "Replaying" 2021-02-20T08:13:04.215 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ + echogrep "state": "Replaying" -i 2021-02-20T08:13:04.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Replaying 2021-02-20T08:13:04.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_state=started 2021-02-20T08:13:04.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test started = started 2021-02-20T08:13:04.217 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:13:04.217 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T08:13:04.218 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:13:04.218 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:13:04.218 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:04.219 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:04.219 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = journal ] 2021-02-20T08:13:04.220 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = snapshot ] 2021-02-20T08:13:04.220 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T08:13:04.220 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:13:04.221 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:13:04.221 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:04.222 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:04.222 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T08:13:04.222 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster2-mirror-test.status+ sed -e s|[/ ]|_|g 2021-02-20T08:13:04.223 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:13:04.223 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:13:04.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T08:13:04.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror-test.status 2021-02-20T08:13:04.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:13:04.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:13:04.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T08:13:04.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:13:04.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:04.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:04.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T08:13:04.228 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: "state": "Replaying" 2021-02-20T08:13:04.333 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:13:05.054 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Snapshot ID: 30 2021-02-20T08:13:05.083 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:13:05.084 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:13:05.084 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:05.084 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:05.085 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:13:05.085 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T08:13:05.085 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:13:05.149 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T08:13:05.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snapshot_id=30 2021-02-20T08:13:05.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:13:05.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.2 2021-02-20T08:13:05.354 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:13:05.354 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:13:05.354 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:13:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:13:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:13:05.436 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:13:05.441 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=26 2021-02-20T08:13:05.442 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 30 = 26 2021-02-20T08:13:05.442 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.4 2021-02-20T08:13:05.843 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:13:05.843 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:13:05.844 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:05.844 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:05.844 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:13:05.844 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:13:05.845 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:13:05.914 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:13:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:13:05.918 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:13:05.918 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:13:05.919 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:13:05.919 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:13:05.920 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 30 = 2021-02-20T08:13:05.920 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.8 2021-02-20T08:13:06.721 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:13:06.721 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:13:06.722 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:06.722 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:06.723 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:13:06.723 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:13:06.723 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:13:06.797 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:13:06.797 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:13:06.801 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:13:06.802 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:13:06.802 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:13:06.803 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:13:06.803 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 30 = 2021-02-20T08:13:06.804 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1.6 2021-02-20T08:13:07.133 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T08:13:07.134 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:13:07.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:13:07.135 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T08:13:07.135 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T08:13:07.136 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T08:13:07.140 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T08:13:07.140 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T08:13:08.403 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:13:08.404 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:13:08.404 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:08.404 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:08.404 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:13:08.405 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:13:08.405 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:13:08.480 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:13:08.480 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:13:08.483 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:13:08.483 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:13:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:13:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:13:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 30 = 2021-02-20T08:13:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 2 2021-02-20T08:13:10.485 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:13:10.486 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:13:10.486 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:10.486 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:10.487 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:13:10.487 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:13:10.487 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:13:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:13:10.572 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=30 2021-02-20T08:13:10.573 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 30 = 30 2021-02-20T08:13:10.573 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:13:10.574 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snap_present cluster1 mirror test snap7 2021-02-20T08:13:10.574 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:13:10.574 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:10.575 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:10.575 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap7 2021-02-20T08:13:10.575 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:13:10.576 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:13:11.574 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 info mirror/test@snap7 2021-02-20T08:13:11.642 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:rbd image 'test': 2021-02-20T08:13:11.642 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: size 512 MiB in 128 objects 2021-02-20T08:13:11.642 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: order 22 (4 MiB objects) 2021-02-20T08:13:11.643 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: snapshot_count: 14 2021-02-20T08:13:11.643 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: id: 10a0aa806784 2021-02-20T08:13:11.643 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: block_name_prefix: rbd_data.10a0aa806784 2021-02-20T08:13:11.643 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: format: 2 2021-02-20T08:13:11.644 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: features: layering, exclusive-lock, object-map, fast-diff, non-primary 2021-02-20T08:13:11.644 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: op_features: 2021-02-20T08:13:11.644 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: flags: 2021-02-20T08:13:11.644 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: create_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:13:11.644 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: access_timestamp: Sat Feb 20 08:12:20 2021 2021-02-20T08:13:11.645 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: modify_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:13:11.645 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: protected: False 2021-02-20T08:13:11.645 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring state: enabled 2021-02-20T08:13:11.645 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring mode: snapshot 2021-02-20T08:13:11.645 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring global id: 3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd 2021-02-20T08:13:11.646 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring primary: false 2021-02-20T08:13:11.647 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:13:11.648 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snap6-clean ] 2021-02-20T08:13:11.648 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap6-clean 2021-02-20T08:13:11.648 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:11.648 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:11.648 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap6-clean 2021-02-20T08:13:11.649 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:13:11.649 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:13:11.649 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:13:11.649 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:13:11.650 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6-clean /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:13:12.285 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:13:12.289 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6-clean /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:13:13.136 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:13:13.140 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:13:13.413 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:13:13.537 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:13:13.538 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap7 2021-02-20T08:13:13.538 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:13.538 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:13.539 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap7 2021-02-20T08:13:13.539 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:13:13.539 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:13:13.539 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:13:13.539 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:13:13.540 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7 /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:13:14.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:13:14.220 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7 /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:13:15.046 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:13:15.050 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:13:15.325 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:13:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:13:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ clean_snap_name=snap7-clean 2021-02-20T08:13:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_snap cluster2 mirror test snap7-clean 2021-02-20T08:13:15.454 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:13:15.454 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:15.454 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:15.454 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap7-clean 2021-02-20T08:13:15.455 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap7-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T08:13:16.070 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T08:13:16.070 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:13:16.071 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:16.071 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:16.071 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:13:16.072 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=11 2021-02-20T08:13:16.072 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:13:16.072 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T08:13:16.716 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 12083200 bytes 2021-02-20T08:13:17.108 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 46998528 bytes 2021-02-20T08:13:17.359 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 69406720 bytes 2021-02-20T08:13:17.588 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 100164608 bytes 2021-02-20T08:13:17.875 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 162909184 bytes 2021-02-20T08:13:18.019 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 172341248 bytes 2021-02-20T08:13:18.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 212862976 bytes 2021-02-20T08:13:18.409 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 224519168 bytes 2021-02-20T08:13:18.679 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 298008576 bytes 2021-02-20T08:13:19.012 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 375617536 bytes 2021-02-20T08:13:19.194 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 394267648 bytes 2021-02-20T08:13:19.434 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 443383808 bytes 2021-02-20T08:13:19.623 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 493753344 bytes 2021-02-20T08:13:19.862 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 515805184 bytes 2021-02-20T08:13:20.154 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 613405696 bytes 2021-02-20T08:13:20.552 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 708696064 bytes 2021-02-20T08:13:20.744 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 732474368 bytes 2021-02-20T08:13:21.013 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 790298624 bytes 2021-02-20T08:13:21.237 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 827801600 bytes 2021-02-20T08:13:21.384 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 831331328 bytes 2021-02-20T08:13:21.565 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 849920000 bytes 2021-02-20T08:13:21.825 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 912956416 bytes 2021-02-20T08:13:22.059 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 955893760 bytes 2021-02-20T08:13:22.600 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 1034587136 bytes 2021-02-20T08:13:22.861 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 1087058944 bytes 2021-02-20T08:13:23.052 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 1108626432 bytes 2021-02-20T08:13:23.234 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2700 IOs, 1129668608 bytes 2021-02-20T08:13:23.433 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2800 IOs, 1158356992 bytes 2021-02-20T08:13:23.679 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2900 IOs, 1210411008 bytes 2021-02-20T08:13:23.958 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3000 IOs, 1267165184 bytes 2021-02-20T08:13:24.316 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3100 IOs, 1281579008 bytes 2021-02-20T08:13:24.531 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3200 IOs, 1321700352 bytes 2021-02-20T08:13:24.701 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3300 IOs, 1326649344 bytes 2021-02-20T08:13:24.931 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3400 IOs, 1364481024 bytes 2021-02-20T08:13:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3500 IOs, 1446641664 bytes 2021-02-20T08:13:25.500 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3600 IOs, 1496115200 bytes 2021-02-20T08:13:25.766 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3700 IOs, 1559847936 bytes 2021-02-20T08:13:25.974 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3800 IOs, 1596683264 bytes 2021-02-20T08:13:26.278 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3900 IOs, 1664568320 bytes 2021-02-20T08:13:26.457 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4000 IOs, 1679611904 bytes 2021-02-20T08:13:26.667 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4100 IOs, 1710406656 bytes 2021-02-20T08:13:26.955 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4200 IOs, 1798458368 bytes 2021-02-20T08:13:27.077 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:13:27.078 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:13:27.078 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:13:27.078 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:13:27.079 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:13:27.079 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:13:27.079 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:27.079 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:27.079 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:13:27.080 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local description_pattern= 2021-02-20T08:13:27.080 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local service_pattern= 2021-02-20T08:13:27.080 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:13:28.082 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:13:28.083 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:13:28.083 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:28.083 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:28.083 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:13:28.083 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local description_pattern= 2021-02-20T08:13:28.084 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local service_pattern= 2021-02-20T08:13:28.084 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T08:13:28.084 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T08:13:28.084 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:13:28.085 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:13:28.085 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ + tee /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:13:28.086 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T08:13:28.163 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:test: 2021-02-20T08:13:28.163 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: global_id: 3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd 2021-02-20T08:13:28.164 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state: up+replaying 2021-02-20T08:13:28.164 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":501172224.0,"local_snapshot_timestamp":1613808732,"remote_snapshot_timestamp":1613808732,"replay_state":"idle"} 2021-02-20T08:13:28.164 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: service: mirror.0 on smithi090 2021-02-20T08:13:28.165 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: last_update: 2021-02-20 08:13:03 2021-02-20T08:13:28.165 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: peer_sites: 2021-02-20T08:13:28.165 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: name: cluster2 2021-02-20T08:13:28.166 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state: up+stopped 2021-02-20T08:13:28.166 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: description: local image is primary 2021-02-20T08:13:28.167 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: last_update: 2021-02-20 08:13:04 2021-02-20T08:13:28.174 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:13:28.176 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: state: up+replaying 2021-02-20T08:13:28.176 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^ description: .* /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:13:28.177 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":501172224.0,"local_snapshot_timestamp":1613808732,"remote_snapshot_timestamp":1613808732,"replay_state":"idle"} 2021-02-20T08:13:28.178 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:13:28.180 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo up+replaying 2021-02-20T08:13:28.180 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^up+ 2021-02-20T08:13:28.180 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep service: *mirror..* on /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:13:28.181 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:up+replaying 2021-02-20T08:13:28.182 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: service: mirror.0 on smithi090 2021-02-20T08:13:28.182 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:13:28.186 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local last_update=2021-02-20 08:13:03 2021-02-20T08:13:28.186 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 08:13:03 2021-02-20T08:13:28.186 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:13:28.187 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:28.187 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:28.188 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:13:28.188 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local prev_last_update=2021-02-20 08:13:03 2021-02-20T08:13:28.188 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T08:13:28.189 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T08:13:28.189 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:13:28.190 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror.mirror_status 2021-02-20T08:13:28.190 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T08:13:28.232 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T08:13:28.295 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local last_update state 2021-02-20T08:13:28.295 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T08:13:28.300 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ last_update=2021-02-20 08:13:03 2021-02-20T08:13:28.301 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T08:13:28.306 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ state=up+replaying 2021-02-20T08:13:28.306 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo up+replaying 2021-02-20T08:13:28.306 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep up+replaying 2021-02-20T08:13:28.307 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:up+replaying 2021-02-20T08:13:28.308 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:13:28.308 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:13:28.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap8 2021-02-20T08:13:28.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_snap cluster2 mirror test snap8 2021-02-20T08:13:28.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:13:28.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:28.310 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:28.310 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap8 2021-02-20T08:13:28.310 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap8 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:13:58.085 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T08:13:58.086 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:13:58.086 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:58.086 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:58.087 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T08:13:58.087 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:13:58.087 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:58.087 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:58.087 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state=started 2021-02-20T08:13:58.088 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:13:58.088 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:13:59.087 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T08:13:59.087 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:13:59.088 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:59.088 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:59.088 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local test_state=started 2021-02-20T08:13:59.088 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_result 2021-02-20T08:13:59.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_state=stopped 2021-02-20T08:13:59.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T08:13:59.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_instance=cluster1 2021-02-20T08:13:59.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ + shift 2021-02-20T08:13:59.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:13:59.090 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:grep -i+ local state instance=cluster1 2021-02-20T08:13:59.090 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:13:59.090 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loop_instance 2021-02-20T08:13:59.090 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0 2021-02-20T08:13:59.094 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:13:59.094 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ seq 0 3 2021-02-20T08:13:59.094 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:13:59.095 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:0 2021-02-20T08:13:59.097 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:13:59.098 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:13:59.098 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:13:59.099 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:13:59.099 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:13:59.099 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:13:59.099 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:13:59.099 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:13:59.100 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:13:59.100 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:13:59.100 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:13:59.100 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:13:59.100 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:13:59.101 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:13:59.101 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:13:59.102 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:13:59.102 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:13:59.102 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:13:59.103 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:13:59.103 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:13:59.104 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:13:59.104 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:13:59.104 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:13:59.104 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:13:59.105 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:13:59.105 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:13:59.105 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:13:59.105 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:13:59.106 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:13:59.110 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:13:59.110 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:13:59.110 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:13:59.111 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:13:59.221 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:13:59.228 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ status_result= "state": "Replaying" 2021-02-20T08:13:59.229 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ + grep -i Replaying 2021-02-20T08:13:59.229 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:echo "state": "Replaying" 2021-02-20T08:13:59.229 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_state=started 2021-02-20T08:13:59.229 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test started = started 2021-02-20T08:13:59.231 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:13:59.231 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T08:13:59.233 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:13:59.233 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:13:59.243 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:59.243 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:59.244 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = journal ] 2021-02-20T08:13:59.244 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = snapshot ] 2021-02-20T08:13:59.245 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T08:13:59.246 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:13:59.247 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:13:59.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:59.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:59.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T08:13:59.250 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster2-mirror-test.status 2021-02-20T08:13:59.250 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:13:59.250 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:13:59.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T08:13:59.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror-test.status 2021-02-20T08:13:59.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:13:59.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:13:59.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T08:13:59.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:13:59.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:13:59.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:13:59.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T08:13:59.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: "state": "Replaying" 2021-02-20T08:14:00.110 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Snapshot ID: 34 2021-02-20T08:14:00.137 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:14:00.138 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:14:00.138 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:00.138 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:00.139 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:14:00.140 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T08:14:00.140 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:14:00.194 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T08:14:00.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snapshot_id=34 2021-02-20T08:14:00.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:14:00.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.2 2021-02-20T08:14:00.401 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:14:00.401 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:14:00.401 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:00.402 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:00.402 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:14:00.402 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:14:00.402 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:14:00.481 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:14:00.485 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=30 2021-02-20T08:14:00.486 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 34 = 30 2021-02-20T08:14:00.486 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.4 2021-02-20T08:14:00.888 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:14:00.888 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:14:00.889 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:00.889 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:00.889 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:14:00.889 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:14:00.890 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:14:00.960 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:14:00.961 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:14:00.965 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:14:00.965 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:14:00.966 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:14:00.966 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:14:00.967 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 34 = 2021-02-20T08:14:00.967 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.8 2021-02-20T08:14:01.768 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:14:01.768 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:14:01.768 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:01.769 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:01.769 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:14:01.769 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:14:01.769 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:14:01.839 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:14:01.840 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:14:01.844 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:14:01.845 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:14:01.845 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:14:01.845 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:14:01.846 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 34 = 2021-02-20T08:14:01.846 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1.6 2021-02-20T08:14:03.447 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:14:03.447 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:14:03.447 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:03.448 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:03.448 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:14:03.448 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:14:03.448 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:14:03.588 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:14:03.588 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:14:03.590 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:14:03.590 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:14:03.591 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:14:03.591 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:14:03.591 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 34 = 2021-02-20T08:14:03.591 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 2 2021-02-20T08:14:05.592 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:14:05.592 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:14:05.593 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:05.593 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:05.593 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:14:05.593 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:14:05.594 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:14:05.673 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:14:05.679 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=34 2021-02-20T08:14:05.679 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 34 = 34 2021-02-20T08:14:05.679 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:14:05.679 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snap_present cluster1 mirror test snap8 2021-02-20T08:14:05.680 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:14:05.680 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:05.680 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:05.680 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap8 2021-02-20T08:14:05.680 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:14:05.681 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:14:06.680 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 info mirror/test@snap8 2021-02-20T08:14:06.750 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:rbd image 'test': 2021-02-20T08:14:06.750 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: size 512 MiB in 128 objects 2021-02-20T08:14:06.751 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: order 22 (4 MiB objects) 2021-02-20T08:14:06.751 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: snapshot_count: 16 2021-02-20T08:14:06.751 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: id: 10a0aa806784 2021-02-20T08:14:06.751 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: block_name_prefix: rbd_data.10a0aa806784 2021-02-20T08:14:06.752 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: format: 2 2021-02-20T08:14:06.752 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: features: layering, exclusive-lock, object-map, fast-diff, non-primary 2021-02-20T08:14:06.752 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: op_features: 2021-02-20T08:14:06.752 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: flags: 2021-02-20T08:14:06.753 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: create_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:14:06.753 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: access_timestamp: Sat Feb 20 08:12:20 2021 2021-02-20T08:14:06.753 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: modify_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:14:06.753 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: protected: False 2021-02-20T08:14:06.754 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring state: enabled 2021-02-20T08:14:06.754 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring mode: snapshot 2021-02-20T08:14:06.754 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring global id: 3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd 2021-02-20T08:14:06.755 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring primary: false 2021-02-20T08:14:06.757 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:14:06.757 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snap7-clean ] 2021-02-20T08:14:06.758 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap7-clean 2021-02-20T08:14:06.758 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:06.758 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:06.758 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap7-clean 2021-02-20T08:14:06.759 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:14:06.759 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:14:06.759 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:14:06.759 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:14:06.759 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7-clean /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:14:07.398 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:14:07.403 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7-clean /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:14:08.271 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:14:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:14:08.555 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:14:08.670 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:14:08.670 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap8 2021-02-20T08:14:08.671 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:08.671 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:08.671 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap8 2021-02-20T08:14:08.671 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:14:08.671 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:14:08.672 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:14:08.672 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:14:08.672 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8 /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:14:09.284 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:14:09.289 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8 /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:14:10.070 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:14:10.074 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:14:10.353 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:14:10.482 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:14:10.482 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ clean_snap_name=snap8-clean 2021-02-20T08:14:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_snap cluster2 mirror test snap8-clean 2021-02-20T08:14:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:14:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap8-clean 2021-02-20T08:14:10.484 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap8-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T08:14:11.117 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T08:14:11.117 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:14:11.117 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:11.118 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:11.118 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:14:11.118 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=11 2021-02-20T08:14:11.119 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:14:11.119 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T08:14:12.201 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 134972416 bytes 2021-02-20T08:14:12.641 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 222364672 bytes 2021-02-20T08:14:12.837 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 245370880 bytes 2021-02-20T08:14:13.013 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 251535360 bytes 2021-02-20T08:14:13.331 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 321710080 bytes 2021-02-20T08:14:13.574 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 363565056 bytes 2021-02-20T08:14:13.869 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 432112640 bytes 2021-02-20T08:14:14.098 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 464174080 bytes 2021-02-20T08:14:14.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 465712128 bytes 2021-02-20T08:14:14.498 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 525379584 bytes 2021-02-20T08:14:14.722 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 548399104 bytes 2021-02-20T08:14:14.961 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 593077248 bytes 2021-02-20T08:14:15.108 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 593689600 bytes 2021-02-20T08:14:15.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 594263040 bytes 2021-02-20T08:14:15.433 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 616681472 bytes 2021-02-20T08:14:15.680 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 669735936 bytes 2021-02-20T08:14:16.012 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 755512320 bytes 2021-02-20T08:14:16.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 809355264 bytes 2021-02-20T08:14:16.507 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 848892928 bytes 2021-02-20T08:14:16.706 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 871804928 bytes 2021-02-20T08:14:16.950 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 914003968 bytes 2021-02-20T08:14:17.107 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 919675904 bytes 2021-02-20T08:14:17.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 920282112 bytes 2021-02-20T08:14:17.530 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 975380480 bytes 2021-02-20T08:14:17.701 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 982259712 bytes 2021-02-20T08:14:18.006 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 1050249216 bytes 2021-02-20T08:14:18.304 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2700 IOs, 1125666816 bytes 2021-02-20T08:14:18.587 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2800 IOs, 1191540736 bytes 2021-02-20T08:14:18.960 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2900 IOs, 1302048768 bytes 2021-02-20T08:14:19.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3000 IOs, 1372427264 bytes 2021-02-20T08:14:19.402 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3100 IOs, 1373035520 bytes 2021-02-20T08:14:19.608 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3200 IOs, 1401618432 bytes 2021-02-20T08:14:19.830 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3300 IOs, 1435874304 bytes 2021-02-20T08:14:20.048 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3400 IOs, 1465304064 bytes 2021-02-20T08:14:20.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3500 IOs, 1490496512 bytes 2021-02-20T08:14:20.465 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3600 IOs, 1533769728 bytes 2021-02-20T08:14:20.704 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3700 IOs, 1554508800 bytes 2021-02-20T08:14:20.893 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3800 IOs, 1577653248 bytes 2021-02-20T08:14:21.056 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3900 IOs, 1603748864 bytes 2021-02-20T08:14:21.141 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5023236428612265) > weight (0.25) 2021-02-20T08:14:21.141 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5104946719619466) > weight (0.25) 2021-02-20T08:14:21.141 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5397249961110497) > weight (0.25) 2021-02-20T08:14:21.141 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T08:14:21.142 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T08:14:21.142 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T08:14:21.143 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T08:14:21.143 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:14:21.144 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T08:14:21.138+0000 7f3ff0a06700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 19127) UID: 1000 2021-02-20T08:14:21.337 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4000 IOs, 1634988032 bytes 2021-02-20T08:14:21.544 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:14:21.719 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4100 IOs, 1739433984 bytes 2021-02-20T08:14:21.942 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4200 IOs, 1788447744 bytes 2021-02-20T08:14:22.123 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:14:22.124 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:14:22.124 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:14:22.124 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:14:22.125 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:14:22.125 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:14:22.125 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:22.125 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:22.125 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:14:22.126 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local description_pattern= 2021-02-20T08:14:22.126 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local service_pattern= 2021-02-20T08:14:22.126 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:14:23.125 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:14:23.125 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:14:23.126 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:23.126 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:23.126 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:14:23.126 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local description_pattern= 2021-02-20T08:14:23.126 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local service_pattern= 2021-02-20T08:14:23.127 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T08:14:23.128 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T08:14:23.128 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:14:23.128 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:14:23.129 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T08:14:23.129 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ tee /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:14:23.208 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:test: 2021-02-20T08:14:23.208 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: global_id: 3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd 2021-02-20T08:14:23.208 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state: up+replaying 2021-02-20T08:14:23.208 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":509284352.0,"local_snapshot_timestamp":1613808784,"remote_snapshot_timestamp":1613808784,"replay_state":"idle"} 2021-02-20T08:14:23.209 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: service: mirror.0 on smithi090 2021-02-20T08:14:23.209 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: last_update: 2021-02-20 08:14:03 2021-02-20T08:14:23.209 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: peer_sites: 2021-02-20T08:14:23.209 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: name: cluster2 2021-02-20T08:14:23.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state: down+stopped 2021-02-20T08:14:23.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: description: local image is primary 2021-02-20T08:14:23.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: last_update: 2021-02-20 08:14:04 2021-02-20T08:14:23.218 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:14:23.220 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: state: up+replaying 2021-02-20T08:14:23.221 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^ description: .* /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:14:23.222 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":509284352.0,"local_snapshot_timestamp":1613808784,"remote_snapshot_timestamp":1613808784,"replay_state":"idle"} 2021-02-20T08:14:23.223 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:14:23.223 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo up+replaying 2021-02-20T08:14:23.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^up+ 2021-02-20T08:14:23.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:up+replaying 2021-02-20T08:14:23.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep service: *mirror..* on /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:14:23.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: service: mirror.0 on smithi090 2021-02-20T08:14:23.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:14:23.229 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local last_update=2021-02-20 08:14:03 2021-02-20T08:14:23.229 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 08:14:03 2021-02-20T08:14:23.229 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:14:23.229 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:23.230 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:23.230 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:14:23.230 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local prev_last_update=2021-02-20 08:14:03 2021-02-20T08:14:23.230 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T08:14:23.231 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T08:14:23.231 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:14:23.232 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror.mirror_status 2021-02-20T08:14:23.232 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T08:14:23.274 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T08:14:23.333 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local last_update state 2021-02-20T08:14:23.333 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T08:14:23.339 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ last_update=2021-02-20 08:14:03 2021-02-20T08:14:23.339 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T08:14:23.342 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ state=up+replaying 2021-02-20T08:14:23.343 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo up+replaying 2021-02-20T08:14:23.343 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep up+replaying 2021-02-20T08:14:23.344 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:up+replaying 2021-02-20T08:14:23.344 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:14:23.345 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:14:23.345 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap9 2021-02-20T08:14:23.345 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_snap cluster2 mirror test snap9 2021-02-20T08:14:23.345 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:14:23.345 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:23.346 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:23.346 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap9 2021-02-20T08:14:23.346 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap9 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:14:26.045 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T08:14:26.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T08:14:26.046 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T08:14:26.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T08:14:26.042+0000 7f90571e9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 18975) UID: 1000 2021-02-20T08:14:26.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:14:27.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:14:27.245 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T08:14:27.246 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T08:14:27.246 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T08:14:27.250 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T08:14:27.250 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T08:14:32.047 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T08:14:32.049 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:14:32.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:14:32.050 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T08:14:32.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T08:14:32.051 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T08:14:32.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T08:14:32.055 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T08:14:53.136 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T08:14:53.137 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:14:53.137 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:53.137 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:53.137 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T08:14:53.138 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:14:53.138 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:53.138 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:53.138 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state=started 2021-02-20T08:14:53.138 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:14:53.139 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:14:54.137 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T08:14:54.138 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:14:54.138 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:54.138 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:54.138 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local test_state=started 2021-02-20T08:14:54.139 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_result 2021-02-20T08:14:54.139 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_state=stopped 2021-02-20T08:14:54.140 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test+ 2021-02-20T08:14:54.140 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ greplocal -i cluster_instance=cluster1 state 2021-02-20T08:14:54.140 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:14:54.141 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:14:54.141 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:14:54.141 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance=cluster1 2021-02-20T08:14:54.141 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loop_instance 2021-02-20T08:14:54.141 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0 2021-02-20T08:14:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:14:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ seq 0 3 2021-02-20T08:14:54.143 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:14:54.144 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:0 2021-02-20T08:14:54.144 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:14:54.144 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:14:54.144 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:14:54.145 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:14:54.145 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:14:54.145 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:14:54.145 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:14:54.146 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:14:54.146 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:14:54.146 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:14:54.146 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:14:54.146 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:14:54.147 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:14:54.147 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:14:54.147 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:14:54.147 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:14:54.147 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:14:54.148 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:14:54.148 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:14:54.148 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:14:54.148 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:14:54.149 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:14:54.149 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:14:54.149 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:14:54.149 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:14:54.149 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:14:54.150 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:14:54.150 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:14:54.150 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:14:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:14:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:14:54.163 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:14:54.163 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:14:54.269 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:14:54.269 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:14:54.270 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:admin_socket: invalid command 2021-02-20T08:14:54.272 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T08:14:54.272 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:1 2021-02-20T08:14:54.272 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:14:54.272 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:14:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T08:14:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:1 2021-02-20T08:14:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:14:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:14:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:14:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:14:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:14:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:14:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:14:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=1 2021-02-20T08:14:54.275 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:14:54.275 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T08:14:54.275 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:1 2021-02-20T08:14:54.275 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:14:54.276 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:14:54.276 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T08:14:54.276 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:1 2021-02-20T08:14:54.276 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:14:54.276 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:14:54.277 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:14:54.277 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:14:54.277 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:14:54.277 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:14:54.277 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:14:54.278 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=1 2021-02-20T08:14:54.278 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:14:54.278 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T08:14:54.298 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:14:54.298 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:14:54.299 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:14:54.299 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T08:14:54.395 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:14:54.395 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:14:54.396 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:admin_socket: invalid command 2021-02-20T08:14:54.398 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T08:14:54.399 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:2 2021-02-20T08:14:54.399 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:14:54.399 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:14:54.399 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T08:14:54.400 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:2 2021-02-20T08:14:54.400 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:14:54.400 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:14:54.400 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:14:54.400 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:14:54.401 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:14:54.401 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:14:54.401 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:14:54.401 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=2 2021-02-20T08:14:54.402 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:14:54.402 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T08:14:54.402 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:2 2021-02-20T08:14:54.403 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:14:54.403 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:14:54.403 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T08:14:54.404 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:2 2021-02-20T08:14:54.404 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:14:54.404 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:14:54.404 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:14:54.404 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:14:54.405 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:14:54.405 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:14:54.405 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:14:54.405 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=2 2021-02-20T08:14:54.406 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:14:54.406 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T08:14:54.411 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:14:54.411 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:14:54.411 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:14:54.412 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T08:14:54.515 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:14:54.523 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ status_result= "state": "Replaying" 2021-02-20T08:14:54.524 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep -i Replaying 2021-02-20T08:14:54.524 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo "state": "Replaying" 2021-02-20T08:14:54.525 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_state=started 2021-02-20T08:14:54.525 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test started = started 2021-02-20T08:14:54.526 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:14:54.526 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T08:14:54.526 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:14:54.527 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:14:54.527 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:54.528 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:54.528 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = journal ] 2021-02-20T08:14:54.528 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = snapshot ] 2021-02-20T08:14:54.529 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T08:14:54.529 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:14:54.530 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:14:54.530 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:54.531 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:54.531 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T08:14:54.532 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster2-mirror-test.status 2021-02-20T08:14:54.532 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:14:54.532 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:14:54.533 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T08:14:54.533 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror-test.status 2021-02-20T08:14:54.534 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:14:54.534 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:14:54.534 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T08:14:54.535 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:14:54.535 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:54.536 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:54.536 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T08:14:54.537 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: "state": "Replaying" 2021-02-20T08:14:55.135 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Snapshot ID: 38 2021-02-20T08:14:55.167 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:14:55.168 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:14:55.168 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:55.168 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:55.169 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:14:55.169 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T08:14:55.170 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:14:55.222 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T08:14:55.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snapshot_id=38 2021-02-20T08:14:55.228 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:14:55.228 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.2 2021-02-20T08:14:55.429 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:14:55.429 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:14:55.429 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:14:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:14:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:14:55.510 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:14:55.516 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=34 2021-02-20T08:14:55.516 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 38 = 34 2021-02-20T08:14:55.516 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.4 2021-02-20T08:14:55.918 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:14:55.918 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:14:55.918 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:55.919 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:55.919 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:14:55.919 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:14:55.919 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:14:55.998 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:14:55.999 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:14:56.002 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:14:56.002 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:14:56.003 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:14:56.003 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:14:56.004 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 38 = 2021-02-20T08:14:56.004 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.8 2021-02-20T08:14:56.805 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:14:56.805 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:14:56.805 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:56.806 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:56.806 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:14:56.806 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:14:56.807 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:14:56.875 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:14:56.876 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:14:56.880 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:14:56.881 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:14:56.881 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:14:56.882 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:14:56.882 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 38 = 2021-02-20T08:14:56.882 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1.6 2021-02-20T08:14:58.483 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:14:58.483 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:14:58.483 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:14:58.483 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:14:58.484 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:14:58.484 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:14:58.484 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:14:58.592 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:14:58.592 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:14:58.595 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:14:58.595 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:14:58.596 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:14:58.596 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:14:58.596 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 38 = 2021-02-20T08:14:58.596 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 2 2021-02-20T08:15:00.597 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:15:00.597 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:00.597 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:00.598 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:00.598 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:15:00.598 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:15:00.598 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:15:00.677 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:15:00.682 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=38 2021-02-20T08:15:00.682 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 38 = 38 2021-02-20T08:15:00.683 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:15:00.683 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snap_present cluster1 mirror test snap9 2021-02-20T08:15:00.683 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:00.683 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:00.684 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:00.684 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap9 2021-02-20T08:15:00.684 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:15:00.684 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:15:01.684 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 info mirror/test@snap9 2021-02-20T08:15:01.754 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:rbd image 'test': 2021-02-20T08:15:01.754 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: size 512 MiB in 128 objects 2021-02-20T08:15:01.754 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: order 22 (4 MiB objects) 2021-02-20T08:15:01.755 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: snapshot_count: 18 2021-02-20T08:15:01.755 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: id: 10a0aa806784 2021-02-20T08:15:01.755 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: block_name_prefix: rbd_data.10a0aa806784 2021-02-20T08:15:01.755 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: format: 2 2021-02-20T08:15:01.756 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: features: layering, exclusive-lock, object-map, fast-diff, non-primary 2021-02-20T08:15:01.756 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: op_features: 2021-02-20T08:15:01.756 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: flags: 2021-02-20T08:15:01.756 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: create_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:15:01.756 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: access_timestamp: Sat Feb 20 08:14:07 2021 2021-02-20T08:15:01.757 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: modify_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:15:01.757 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: protected: False 2021-02-20T08:15:01.757 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring state: enabled 2021-02-20T08:15:01.757 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring mode: snapshot 2021-02-20T08:15:01.758 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring global id: 3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd 2021-02-20T08:15:01.758 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring primary: false 2021-02-20T08:15:01.760 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:15:01.760 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snap8-clean ] 2021-02-20T08:15:01.760 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap8-clean 2021-02-20T08:15:01.761 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:01.761 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:01.761 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap8-clean 2021-02-20T08:15:01.761 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:15:01.762 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:15:01.762 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:15:01.762 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:15:01.763 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8-clean /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:15:02.423 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:15:02.428 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8-clean /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:15:03.261 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:15:03.265 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:15:03.544 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:15:03.660 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:15:03.661 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap9 2021-02-20T08:15:03.661 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:03.661 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:03.661 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap9 2021-02-20T08:15:03.662 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:15:03.662 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:15:03.662 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:15:03.662 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:15:03.663 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9 /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:15:04.527 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:15:04.531 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9 /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:15:05.378 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:15:05.381 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:15:05.656 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:15:05.782 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:15:05.782 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ clean_snap_name=snap9-clean 2021-02-20T08:15:05.782 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_snap cluster2 mirror test snap9-clean 2021-02-20T08:15:05.783 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:15:05.783 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:05.783 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:05.783 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap9-clean 2021-02-20T08:15:05.783 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap9-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T08:15:06.038 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T08:15:06.038 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:15:06.039 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:06.039 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:06.039 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:15:06.040 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=7 2021-02-20T08:15:06.040 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:15:06.041 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T08:15:06.895 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 23568384 bytes 2021-02-20T08:15:07.385 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 93603840 bytes 2021-02-20T08:15:07.750 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 163988480 bytes 2021-02-20T08:15:08.011 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 210048000 bytes 2021-02-20T08:15:08.412 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 328061952 bytes 2021-02-20T08:15:08.663 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 368081920 bytes 2021-02-20T08:15:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 478725120 bytes 2021-02-20T08:15:09.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 498723840 bytes 2021-02-20T08:15:09.403 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 499341312 bytes 2021-02-20T08:15:09.662 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 557541376 bytes 2021-02-20T08:15:09.904 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 580736000 bytes 2021-02-20T08:15:10.077 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 602478592 bytes 2021-02-20T08:15:10.372 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 654868480 bytes 2021-02-20T08:15:10.652 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 717786112 bytes 2021-02-20T08:15:10.889 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 747812864 bytes 2021-02-20T08:15:11.106 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 783723520 bytes 2021-02-20T08:15:11.337 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 841821184 bytes 2021-02-20T08:15:11.625 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 883520512 bytes 2021-02-20T08:15:11.889 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 924323840 bytes 2021-02-20T08:15:12.203 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 1006814208 bytes 2021-02-20T08:15:12.513 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 1073098752 bytes 2021-02-20T08:15:12.749 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 1109650432 bytes 2021-02-20T08:15:12.959 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 1130904576 bytes 2021-02-20T08:15:13.043 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:15:13.043 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:15:13.044 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:15:13.044 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:15:13.044 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:15:13.044 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:13.045 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:13.045 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:13.045 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:15:13.045 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local description_pattern= 2021-02-20T08:15:13.045 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local service_pattern= 2021-02-20T08:15:13.046 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:15:14.044 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:15:14.045 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:14.045 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:14.045 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:14.045 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:15:14.045 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local description_pattern= 2021-02-20T08:15:14.046 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local service_pattern= 2021-02-20T08:15:14.046 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T08:15:14.047 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T08:15:14.047 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:15:14.048 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:15:14.048 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T08:15:14.048 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ tee /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:15:14.130 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:test: 2021-02-20T08:15:14.131 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: global_id: 3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd 2021-02-20T08:15:14.131 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state: up+replaying 2021-02-20T08:15:14.131 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"local_snapshot_timestamp":1613808840,"remote_snapshot_timestamp":1613808840,"replay_state":"idle"} 2021-02-20T08:15:14.132 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: service: mirror.2 on smithi090 2021-02-20T08:15:14.132 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: last_update: 2021-02-20 08:14:53 2021-02-20T08:15:14.132 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: peer_sites: 2021-02-20T08:15:14.132 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: name: cluster2 2021-02-20T08:15:14.133 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state: up+stopped 2021-02-20T08:15:14.133 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: description: local image is primary 2021-02-20T08:15:14.133 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: last_update: 2021-02-20 08:14:58 2021-02-20T08:15:14.141 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:15:14.143 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: state: up+replaying 2021-02-20T08:15:14.143 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^ description: .* /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:15:14.145 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"local_snapshot_timestamp":1613808840,"remote_snapshot_timestamp":1613808840,"replay_state":"idle"} 2021-02-20T08:15:14.145 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:15:14.146 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo up+replaying 2021-02-20T08:15:14.146 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep ^up+ 2021-02-20T08:15:14.146 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:up+replaying 2021-02-20T08:15:14.147 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep service: *mirror..* on /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:15:14.148 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: service: mirror.2 on smithi090 2021-02-20T08:15:14.148 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.mirror_status 2021-02-20T08:15:14.151 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local last_update=2021-02-20 08:14:53 2021-02-20T08:15:14.151 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 08:14:53 2021-02-20T08:15:14.151 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:14.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:14.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:14.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state_pattern=up+replaying 2021-02-20T08:15:14.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local prev_last_update=2021-02-20 08:14:53 2021-02-20T08:15:14.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T08:15:14.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T08:15:14.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:15:14.154 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror.mirror_status 2021-02-20T08:15:14.154 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T08:15:14.197 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T08:15:14.259 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local last_update state 2021-02-20T08:15:14.259 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T08:15:14.265 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ last_update=2021-02-20 08:14:53 2021-02-20T08:15:14.265 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T08:15:14.270 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ state=up+replaying 2021-02-20T08:15:14.271 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo up+replaying 2021-02-20T08:15:14.271 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep up+replaying 2021-02-20T08:15:14.272 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:up+replaying 2021-02-20T08:15:14.273 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:15:14.273 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:15:14.273 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap10 2021-02-20T08:15:14.273 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_snap cluster2 mirror test snap10 2021-02-20T08:15:14.274 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:15:14.274 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:14.274 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:14.274 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap10 2021-02-20T08:15:14.275 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap10 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:15:44.175 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T08:15:44.176 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:44.176 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:44.176 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:44.177 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T08:15:44.177 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:44.178 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:44.178 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:44.179 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state=started 2021-02-20T08:15:44.179 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:15:44.179 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:15:45.174 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T08:15:45.175 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:45.175 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:45.175 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:45.176 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local test_state=started 2021-02-20T08:15:45.176 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_result 2021-02-20T08:15:45.176 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_state=stopped 2021-02-20T08:15:45.179 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T08:15:45.179 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_instance=cluster1 2021-02-20T08:15:45.179 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:15:45.179 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:45.179 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ + local instance=cluster1 2021-02-20T08:15:45.180 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loop_instance 2021-02-20T08:15:45.180 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grepsleep -i 0 2021-02-20T08:15:45.180 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: state 2021-02-20T08:15:45.180 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:15:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ seq 0 3 2021-02-20T08:15:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:15:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:0 2021-02-20T08:15:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:15:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:15:45.182 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:15:45.182 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:15:45.182 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:15:45.182 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:15:45.182 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:15:45.183 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:15:45.183 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:15:45.183 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:15:45.183 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:15:45.183 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:15:45.184 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:15:45.184 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:15:45.184 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:15:45.184 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:45.184 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:15:45.185 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:15:45.185 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:15:45.185 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:15:45.185 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:15:45.185 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:15:45.186 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:15:45.186 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:15:45.186 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:15:45.186 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:15:45.187 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:15:45.187 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:15:45.187 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:15:45.203 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:15:45.203 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:15:45.203 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:15:45.204 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:15:45.306 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:15:45.307 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:15:45.307 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:admin_socket: invalid command 2021-02-20T08:15:45.311 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T08:15:45.311 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:1 2021-02-20T08:15:45.311 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:15:45.312 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:15:45.312 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T08:15:45.313 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:1 2021-02-20T08:15:45.313 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:15:45.313 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:15:45.313 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:15:45.314 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:15:45.314 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:15:45.314 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:15:45.314 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:15:45.315 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=1 2021-02-20T08:15:45.315 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:15:45.315 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T08:15:45.315 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:1 2021-02-20T08:15:45.315 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:45.316 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:15:45.316 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T08:15:45.316 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:1 2021-02-20T08:15:45.316 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:15:45.316 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:15:45.317 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:15:45.317 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:15:45.317 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:15:45.317 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:15:45.317 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:15:45.318 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=1 2021-02-20T08:15:45.318 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:15:45.318 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T08:15:45.330 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:15:45.331 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:15:45.331 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:15:45.332 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T08:15:45.432 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:15:45.433 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:15:45.433 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:admin_socket: invalid command 2021-02-20T08:15:45.436 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T08:15:45.437 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:2 2021-02-20T08:15:45.437 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:15:45.437 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:15:45.437 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T08:15:45.437 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:2 2021-02-20T08:15:45.438 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:15:45.438 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:15:45.438 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:15:45.438 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:15:45.439 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:15:45.439 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:15:45.439 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:15:45.439 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=2 2021-02-20T08:15:45.439 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:15:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T08:15:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:2 2021-02-20T08:15:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:15:45.442 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T08:15:45.442 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:2 2021-02-20T08:15:45.442 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:15:45.442 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:15:45.442 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:15:45.443 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:15:45.443 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:15:45.443 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:15:45.443 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:15:45.443 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=2 2021-02-20T08:15:45.444 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:15:45.444 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T08:15:45.450 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:15:45.451 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:15:45.451 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:15:45.451 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T08:15:45.561 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:15:45.562 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ status_result= "state": "Replaying" 2021-02-20T08:15:45.563 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo "state": "Replaying" 2021-02-20T08:15:45.563 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep -i Replaying 2021-02-20T08:15:45.564 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: "state": "Replaying" 2021-02-20T08:15:45.565 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_state=started 2021-02-20T08:15:45.565 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test started = started 2021-02-20T08:15:45.565 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:15:45.565 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T08:15:45.566 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:15:45.566 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:15:45.566 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:45.566 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:45.566 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = journal ] 2021-02-20T08:15:45.567 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = snapshot ] 2021-02-20T08:15:45.567 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T08:15:45.567 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:15:45.567 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:15:45.568 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:45.568 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:45.568 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T08:15:45.569 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster2-mirror-test.status 2021-02-20T08:15:45.569 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:15:45.571 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:15:45.572 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T08:15:45.572 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror-test.status 2021-02-20T08:15:45.572 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:15:45.572 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:15:45.572 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T08:15:45.573 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:15:45.573 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:45.573 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:45.573 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T08:15:46.189 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Snapshot ID: 42 2021-02-20T08:15:46.215 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:15:46.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:15:46.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:46.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:46.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:15:46.217 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T08:15:46.217 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:15:46.275 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T08:15:46.279 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snapshot_id=42 2021-02-20T08:15:46.279 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:15:46.279 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.2 2021-02-20T08:15:46.481 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:15:46.482 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:46.482 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:46.483 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:46.483 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:15:46.484 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:15:46.484 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:15:46.563 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:15:46.569 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=38 2021-02-20T08:15:46.570 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 42 = 38 2021-02-20T08:15:46.570 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.4 2021-02-20T08:15:46.970 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:15:46.971 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:46.971 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:46.971 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:46.971 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:15:46.972 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:15:46.972 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:15:47.041 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:15:47.041 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:15:47.045 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:15:47.045 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:15:47.045 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:15:47.046 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:15:47.046 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 42 = 2021-02-20T08:15:47.047 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.8 2021-02-20T08:15:47.848 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:15:47.848 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:47.848 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:47.849 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:47.849 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:15:47.849 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:15:47.849 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:15:47.936 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:15:47.937 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:15:47.941 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:15:47.941 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:15:47.942 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:15:47.942 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:15:47.943 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 42 = 2021-02-20T08:15:47.943 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1.6 2021-02-20T08:15:49.544 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:15:49.544 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:49.544 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:49.545 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:49.545 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:15:49.545 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:15:49.545 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:15:49.640 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:15:49.640 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:15:49.643 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:15:49.643 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:15:49.643 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:15:49.644 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:15:49.644 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 42 = 2021-02-20T08:15:49.645 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 2 2021-02-20T08:15:51.646 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:15:51.646 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:51.647 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:51.647 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:51.647 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:15:51.648 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:15:51.648 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:15:51.734 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:15:51.739 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=42 2021-02-20T08:15:51.740 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 42 = 42 2021-02-20T08:15:51.740 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:15:51.741 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snap_present cluster1 mirror test snap10 2021-02-20T08:15:51.741 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:51.741 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:51.741 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:51.742 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap10 2021-02-20T08:15:51.742 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:15:51.742 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:15:52.741 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 info mirror/test@snap10 2021-02-20T08:15:52.811 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:rbd image 'test': 2021-02-20T08:15:52.812 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: size 512 MiB in 128 objects 2021-02-20T08:15:52.812 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: order 22 (4 MiB objects) 2021-02-20T08:15:52.812 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: snapshot_count: 20 2021-02-20T08:15:52.812 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: id: 10a0aa806784 2021-02-20T08:15:52.813 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: block_name_prefix: rbd_data.10a0aa806784 2021-02-20T08:15:52.813 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: format: 2 2021-02-20T08:15:52.813 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: features: layering, exclusive-lock, object-map, fast-diff, non-primary 2021-02-20T08:15:52.813 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: op_features: 2021-02-20T08:15:52.813 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: flags: 2021-02-20T08:15:52.814 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: create_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:15:52.814 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: access_timestamp: Sat Feb 20 08:14:07 2021 2021-02-20T08:15:52.814 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: modify_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:15:52.814 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: protected: False 2021-02-20T08:15:52.815 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring state: enabled 2021-02-20T08:15:52.815 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring mode: snapshot 2021-02-20T08:15:52.815 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring global id: 3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd 2021-02-20T08:15:52.815 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring primary: false 2021-02-20T08:15:52.817 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:15:52.818 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snap9-clean ] 2021-02-20T08:15:52.818 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap9-clean 2021-02-20T08:15:52.818 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:52.818 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:52.819 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap9-clean 2021-02-20T08:15:52.819 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:15:52.819 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:15:52.820 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:15:52.820 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:15:52.820 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9-clean /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:15:53.471 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:15:53.475 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9-clean /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:15:54.139 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:15:54.143 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:15:54.420 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:15:54.537 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:15:54.538 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap10 2021-02-20T08:15:54.538 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:54.538 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:54.539 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap10 2021-02-20T08:15:54.539 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:15:54.539 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:15:54.539 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:15:54.540 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:15:54.540 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap10 /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:15:55.218 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:15:55.222 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap10 /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:15:56.036 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:15:56.041 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:15:56.319 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:15:56.438 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:15:56.438 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ clean_snap_name=snap10-clean 2021-02-20T08:15:56.438 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_snap cluster2 mirror test snap10-clean 2021-02-20T08:15:56.438 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:15:56.439 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:56.439 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:56.439 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap10-clean 2021-02-20T08:15:56.439 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap10-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T08:15:57.087 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T08:15:57.087 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:57.088 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:57.088 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:57.088 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T08:15:57.088 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:57.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:57.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:57.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state=started 2021-02-20T08:15:57.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:15:57.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:15:58.088 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T08:15:58.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:58.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:58.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:58.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local test_state=started 2021-02-20T08:15:58.090 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_result 2021-02-20T08:15:58.090 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_state=stopped 2021-02-20T08:15:58.092 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T08:15:58.092 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_instance=cluster1 2021-02-20T08:15:58.093 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ + shift 2021-02-20T08:15:58.093 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:58.093 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:grep -i state+ 2021-02-20T08:15:58.093 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:local instance=cluster1 2021-02-20T08:15:58.093 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loop_instance 2021-02-20T08:15:58.094 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0 2021-02-20T08:15:58.094 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:15:58.094 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ seq 0 3 2021-02-20T08:15:58.094 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:15:58.095 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:0 2021-02-20T08:15:58.095 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:15:58.095 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:15:58.095 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:15:58.095 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:15:58.096 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:15:58.096 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:15:58.096 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:15:58.096 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:15:58.096 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:15:58.097 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:15:58.097 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:15:58.097 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:15:58.097 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:15:58.098 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:15:58.098 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:15:58.098 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:58.098 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:15:58.099 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:15:58.099 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:0 2021-02-20T08:15:58.100 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:15:58.100 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:15:58.100 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:15:58.100 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:15:58.101 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:15:58.101 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:15:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:15:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=0 2021-02-20T08:15:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=0 2021-02-20T08:15:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:15:58.115 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:15:58.115 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:15:58.115 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:15:58.115 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:15:58.218 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:15:58.218 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:15:58.219 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:admin_socket: invalid command 2021-02-20T08:15:58.221 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T08:15:58.221 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:1 2021-02-20T08:15:58.221 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:15:58.222 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:15:58.222 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T08:15:58.222 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:1 2021-02-20T08:15:58.222 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:15:58.223 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:15:58.223 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:15:58.223 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:15:58.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:15:58.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:15:58.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:15:58.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=1 2021-02-20T08:15:58.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:15:58.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T08:15:58.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:1 2021-02-20T08:15:58.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:58.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:15:58.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T08:15:58.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:1 2021-02-20T08:15:58.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:15:58.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:15:58.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:15:58.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:15:58.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:15:58.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:15:58.228 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:15:58.228 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=1 2021-02-20T08:15:58.228 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=1 2021-02-20T08:15:58.228 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T08:15:58.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:15:58.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:15:58.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:15:58.241 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T08:15:58.345 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:15:58.346 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:15:58.346 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:admin_socket: invalid command 2021-02-20T08:15:58.350 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T08:15:58.351 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1:2 2021-02-20T08:15:58.351 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:15:58.351 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:15:58.351 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T08:15:58.352 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:2 2021-02-20T08:15:58.352 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=cluster 2021-02-20T08:15:58.352 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:15:58.352 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:15:58.352 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:15:58.353 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:15:58.353 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval cluster=cluster1 2021-02-20T08:15:58.353 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:15:58.353 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=2 2021-02-20T08:15:58.354 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:15:58.354 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T08:15:58.354 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1:2 2021-02-20T08:15:58.354 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:58.354 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance 2021-02-20T08:15:58.355 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T08:15:58.355 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local val=cluster1:2 2021-02-20T08:15:58.355 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:15:58.355 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local instance_var_name=instance 2021-02-20T08:15:58.355 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cluster=cluster1 2021-02-20T08:15:58.356 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:15:58.356 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:15:58.356 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval local_cluster=cluster1 2021-02-20T08:15:58.356 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local_cluster=cluster1 2021-02-20T08:15:58.357 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ eval instance=2 2021-02-20T08:15:58.357 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ instance=2 2021-02-20T08:15:58.357 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T08:15:58.368 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:15:58.368 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:15:58.369 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:15:58.369 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T08:15:58.470 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:15:58.471 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ status_result= "state": "Replaying" 2021-02-20T08:15:58.472 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo "state": "Replaying" 2021-02-20T08:15:58.472 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ grep -i Replaying 2021-02-20T08:15:58.473 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: "state": "Replaying" 2021-02-20T08:15:58.473 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_state=started 2021-02-20T08:15:58.474 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test started = started 2021-02-20T08:15:58.474 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:15:58.474 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T08:15:58.474 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:15:58.475 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:15:58.475 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:58.475 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:58.475 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = journal ] 2021-02-20T08:15:58.475 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ snapshot = snapshot ] 2021-02-20T08:15:58.476 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T08:15:58.476 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_cluster=cluster1 2021-02-20T08:15:58.476 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:15:58.476 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:58.476 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:58.477 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T08:15:58.477 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ + echo cluster2-mirror-test.status 2021-02-20T08:15:58.477 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:sed -e s|[/ ]|_|g 2021-02-20T08:15:58.478 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local status_log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:15:58.479 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T08:15:58.479 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo cluster1-mirror-test.status 2021-02-20T08:15:58.479 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:15:58.479 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local local_status_log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:15:58.480 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T08:15:58.480 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:15:58.480 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:58.480 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:58.480 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T08:15:59.099 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Snapshot ID: 45 2021-02-20T08:15:59.111 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:15:59.111 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:15:59.111 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:59.112 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:59.112 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.status 2021-02-20T08:15:59.112 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T08:15:59.112 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:15:59.387 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T08:15:59.391 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snapshot_id=45 2021-02-20T08:15:59.391 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:15:59.391 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.2 2021-02-20T08:15:59.593 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:15:59.594 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:15:59.594 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:15:59.594 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:15:59.595 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:15:59.595 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:15:59.595 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:15:59.671 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:15:59.677 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=42 2021-02-20T08:15:59.677 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 45 = 42 2021-02-20T08:15:59.677 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.4 2021-02-20T08:16:00.079 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:16:00.079 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:16:00.080 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:00.080 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:00.080 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:16:00.080 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:16:00.080 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:16:00.156 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:16:00.156 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:16:00.161 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:16:00.161 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:16:00.161 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:16:00.162 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:16:00.162 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 45 = 2021-02-20T08:16:00.163 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.8 2021-02-20T08:16:00.963 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:16:00.964 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:16:00.964 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:00.964 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:00.964 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:16:00.965 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:16:00.965 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:16:01.054 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ true 2021-02-20T08:16:01.055 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:16:01.059 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:-:1.1: Document is empty 2021-02-20T08:16:01.060 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: 2021-02-20T08:16:01.060 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:^ 2021-02-20T08:16:01.060 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id= 2021-02-20T08:16:01.061 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 45 = 2021-02-20T08:16:01.061 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1.6 2021-02-20T08:16:02.662 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:16:02.662 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:16:02.663 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:02.663 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:02.663 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local log=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.status 2021-02-20T08:16:02.663 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:16:02.663 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:16:02.742 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:16:02.748 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local primary_snapshot_id=45 2021-02-20T08:16:02.748 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test 45 = 45 2021-02-20T08:16:02.749 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:02.749 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_snap_present cluster1 mirror test snap10-clean 2021-02-20T08:16:02.749 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:16:02.749 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:02.749 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:02.750 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap10-clean 2021-02-20T08:16:02.750 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:16:02.750 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:16:03.749 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 info mirror/test@snap10-clean 2021-02-20T08:16:03.820 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:rbd image 'test': 2021-02-20T08:16:03.820 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: size 512 MiB in 128 objects 2021-02-20T08:16:03.821 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: order 22 (4 MiB objects) 2021-02-20T08:16:03.821 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: snapshot_count: 21 2021-02-20T08:16:03.821 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: id: 10a0aa806784 2021-02-20T08:16:03.821 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: block_name_prefix: rbd_data.10a0aa806784 2021-02-20T08:16:03.821 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: format: 2 2021-02-20T08:16:03.822 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: features: layering, exclusive-lock, object-map, fast-diff, non-primary 2021-02-20T08:16:03.822 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: op_features: 2021-02-20T08:16:03.822 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: flags: 2021-02-20T08:16:03.823 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: create_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:16:03.823 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: access_timestamp: Sat Feb 20 08:15:53 2021 2021-02-20T08:16:03.823 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: modify_timestamp: Sat Feb 20 08:07:06 2021 2021-02-20T08:16:03.824 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: protected: False 2021-02-20T08:16:03.824 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring state: enabled 2021-02-20T08:16:03.825 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring mode: snapshot 2021-02-20T08:16:03.825 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring global id: 3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd 2021-02-20T08:16:03.826 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout: mirroring primary: false 2021-02-20T08:16:03.828 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:03.828 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ seq 1 10 2021-02-20T08:16:03.829 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap1 2021-02-20T08:16:03.829 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap1 2021-02-20T08:16:03.830 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:03.830 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:03.830 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap1 2021-02-20T08:16:03.831 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:16:03.831 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:03.831 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:03.831 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:03.832 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1 /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:04.472 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:04.476 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1 /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:05.232 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:05.237 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:05.517 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:05.647 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:05.647 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap1-clean 2021-02-20T08:16:05.648 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap1-clean 2021-02-20T08:16:05.648 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:05.649 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:05.649 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap1-clean 2021-02-20T08:16:05.649 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:16:05.650 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:05.650 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:05.651 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:05.651 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1-clean /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:06.057 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T08:16:06.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T08:16:06.058 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T08:16:06.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T08:16:06.053+0000 7f8668f31700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 25192) UID: 1000 2021-02-20T08:16:06.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:16:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:06.296 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1-clean /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:07.140 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:07.143 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:07.422 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:07.537 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:07.538 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap2 2021-02-20T08:16:07.538 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap2 2021-02-20T08:16:07.538 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:07.538 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:07.539 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap2 2021-02-20T08:16:07.539 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:16:07.539 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:07.539 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:07.539 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:07.540 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2 /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:08.173 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:08.178 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2 /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:09.013 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:09.017 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:09.292 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:09.419 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:09.419 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap2-clean 2021-02-20T08:16:09.419 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap2-clean 2021-02-20T08:16:09.420 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:09.420 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:09.420 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap2-clean 2021-02-20T08:16:09.420 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:16:09.421 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:09.421 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:09.421 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:09.422 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2-clean /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:10.062 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:10.066 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2-clean /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:10.921 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:10.924 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:11.196 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:11.324 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:11.324 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap3 2021-02-20T08:16:11.325 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap3 2021-02-20T08:16:11.325 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:11.325 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:11.326 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap3 2021-02-20T08:16:11.326 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:16:11.327 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:11.327 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:11.327 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:11.328 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3 /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:11.974 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:11.981 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3 /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:12.059 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T08:16:12.059 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:16:12.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:16:12.060 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T08:16:12.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T08:16:12.061 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T08:16:12.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T08:16:12.081 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T08:16:12.852 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:12.857 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:13.139 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:13.266 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:13.267 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap3-clean 2021-02-20T08:16:13.267 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap3-clean 2021-02-20T08:16:13.267 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:13.267 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:13.267 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap3-clean 2021-02-20T08:16:13.268 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:16:13.268 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:13.268 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:13.268 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:13.269 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3-clean /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:13.922 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:13.925 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3-clean /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:14.672 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:14.675 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:14.954 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:15.084 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:15.086 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap4 2021-02-20T08:16:15.087 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap4 2021-02-20T08:16:15.087 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:15.087 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:15.088 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap4 2021-02-20T08:16:15.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:16:15.090 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:15.091 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:15.091 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:15.093 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4 /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:15.699 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:15.703 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4 /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:16.493 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:16.496 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:16.773 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:16.893 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:16.894 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap4-clean 2021-02-20T08:16:16.894 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap4-clean 2021-02-20T08:16:16.894 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:16.894 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:16.895 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap4-clean 2021-02-20T08:16:16.895 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:16:16.895 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:16.895 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:16.896 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:16.896 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4-clean /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:17.533 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:17.537 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4-clean /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:18.335 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:18.339 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:18.620 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:18.748 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:18.748 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap5 2021-02-20T08:16:18.749 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap5 2021-02-20T08:16:18.749 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:18.749 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:18.749 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap5 2021-02-20T08:16:18.750 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:16:18.750 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:18.750 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:18.750 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:18.751 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5 /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:19.403 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:19.407 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5 /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:20.251 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3399625062035302) > weight (0.25) 2021-02-20T08:16:20.252 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7193066104439357) > weight (0.25) 2021-02-20T08:16:20.252 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.37902822204915243) > weight (0.25) 2021-02-20T08:16:20.253 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8869831731216484) > weight (0.25) 2021-02-20T08:16:20.253 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T08:16:20.283 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:20.286 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:20.564 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:20.692 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:20.693 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap5-clean 2021-02-20T08:16:20.693 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap5-clean 2021-02-20T08:16:20.693 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:20.694 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:20.694 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap5-clean 2021-02-20T08:16:20.694 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:16:20.695 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:20.695 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:20.695 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:20.696 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5-clean /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:21.348 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:21.352 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5-clean /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:22.191 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:22.196 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:22.482 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:22.601 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:22.601 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap6 2021-02-20T08:16:22.602 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap6 2021-02-20T08:16:22.602 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:22.602 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:22.603 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap6 2021-02-20T08:16:22.603 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:16:22.603 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:22.603 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:22.604 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:22.604 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6 /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:23.233 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:23.237 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6 /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:24.097 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:24.100 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:24.376 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:24.505 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:24.506 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap6-clean 2021-02-20T08:16:24.506 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap6-clean 2021-02-20T08:16:24.506 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:24.507 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:24.507 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap6-clean 2021-02-20T08:16:24.507 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:16:24.507 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:24.508 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:24.508 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:24.508 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6-clean /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:25.131 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:25.135 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6-clean /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:25.974 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:25.978 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:26.256 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:26.385 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:26.386 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap7 2021-02-20T08:16:26.386 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap7 2021-02-20T08:16:26.386 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:26.386 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:26.387 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap7 2021-02-20T08:16:26.387 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:16:26.388 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:26.388 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:26.388 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:26.389 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7 /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:27.033 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:27.037 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7 /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:27.830 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:27.834 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:28.112 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:28.241 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:28.241 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap7-clean 2021-02-20T08:16:28.241 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap7-clean 2021-02-20T08:16:28.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:28.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:28.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap7-clean 2021-02-20T08:16:28.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:16:28.243 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:28.243 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:28.243 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:28.244 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7-clean /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:28.915 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:28.920 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7-clean /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:29.700 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:29.704 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:29.978 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:30.108 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:30.109 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap8 2021-02-20T08:16:30.109 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap8 2021-02-20T08:16:30.109 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:30.110 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:30.110 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap8 2021-02-20T08:16:30.110 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:16:30.110 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:30.110 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:30.111 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:30.111 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8 /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:30.775 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:30.792 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8 /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:31.585 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:31.590 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:31.870 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:32.000 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:32.001 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap8-clean 2021-02-20T08:16:32.001 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap8-clean 2021-02-20T08:16:32.001 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:32.001 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:32.002 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap8-clean 2021-02-20T08:16:32.002 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:16:32.002 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:32.002 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:32.002 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:32.003 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8-clean /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:32.668 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:32.671 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8-clean /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:33.521 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:33.527 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:33.802 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:33.932 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:33.933 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap9 2021-02-20T08:16:33.933 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap9 2021-02-20T08:16:33.933 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:33.933 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:33.933 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap9 2021-02-20T08:16:33.934 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:16:33.934 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:33.934 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:33.934 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:33.935 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9 /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:34.598 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:34.604 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9 /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:35.423 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:35.427 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:35.706 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:35.836 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:35.837 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap9-clean 2021-02-20T08:16:35.838 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap9-clean 2021-02-20T08:16:35.838 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:35.838 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:35.839 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap9-clean 2021-02-20T08:16:35.839 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:16:35.840 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:35.840 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:35.841 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:35.842 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9-clean /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:36.502 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:36.508 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9-clean /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:37.330 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:37.332 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:37.612 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:37.746 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:37.747 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap10 2021-02-20T08:16:37.747 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap10 2021-02-20T08:16:37.747 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:37.747 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:37.748 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap10 2021-02-20T08:16:37.748 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:16:37.748 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:37.748 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:37.749 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:37.749 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap10 /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:38.400 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:38.403 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap10 /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:39.188 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:39.193 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:39.473 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:39.603 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap10-clean 2021-02-20T08:16:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ compare_image_snaps mirror test snap10-clean 2021-02-20T08:16:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:39.606 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap_name=snap10-clean 2021-02-20T08:16:39.606 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local ret=0 2021-02-20T08:16:39.607 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local rmt_export=/tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:39.607 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local loc_export=/tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:39.608 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:39.608 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 -p mirror export test@snap10-clean /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export 2021-02-20T08:16:40.258 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:40.262 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster1 -p mirror export test@snap10-clean /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:41.053 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:16:41.056 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ cmp /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:41.336 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rm -f /tmp/tmp.1RV4oXZGap/cluster2-mirror-test.export /tmp/tmp.1RV4oXZGap/cluster1-mirror-test.export 2021-02-20T08:16:41.466 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:16:41.466 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ seq 1 10 2021-02-20T08:16:41.467 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap1 2021-02-20T08:16:41.467 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_snapshot cluster2 mirror test snap1 2021-02-20T08:16:41.467 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:16:41.468 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:41.468 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:41.468 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap=snap1 2021-02-20T08:16:41.468 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap1 2021-02-20T08:16:42.122 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Removing snap: 100% complete...done. 2021-02-20T08:16:42.138 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap1-clean 2021-02-20T08:16:42.138 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_snapshot cluster2 mirror test snap1-clean 2021-02-20T08:16:42.139 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:16:42.139 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:42.139 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:42.139 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap=snap1-clean 2021-02-20T08:16:42.140 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap1-clean 2021-02-20T08:16:43.122 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Removing snap: 100% complete...done. 2021-02-20T08:16:43.157 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap2 2021-02-20T08:16:43.157 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_snapshot cluster2 mirror test snap2 2021-02-20T08:16:43.157 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:16:43.157 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:43.158 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:43.158 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap=snap2 2021-02-20T08:16:43.158 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap2 2021-02-20T08:16:44.126 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Removing snap: 100% complete...done. 2021-02-20T08:16:44.149 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap2-clean 2021-02-20T08:16:44.149 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_snapshot cluster2 mirror test snap2-clean 2021-02-20T08:16:44.150 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:16:44.150 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:44.151 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:44.151 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap=snap2-clean 2021-02-20T08:16:44.151 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap2-clean 2021-02-20T08:16:45.144 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Removing snap: 100% complete...done. 2021-02-20T08:16:45.168 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap3 2021-02-20T08:16:45.168 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_snapshot cluster2 mirror test snap3 2021-02-20T08:16:45.168 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:16:45.169 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:45.169 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:45.169 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap=snap3 2021-02-20T08:16:45.169 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap3 2021-02-20T08:16:46.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Removing snap: 100% complete...done. 2021-02-20T08:16:46.168 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap3-clean 2021-02-20T08:16:46.168 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_snapshot cluster2 mirror test snap3-clean 2021-02-20T08:16:46.169 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:16:46.169 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:46.169 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:46.169 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap=snap3-clean 2021-02-20T08:16:46.169 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap3-clean 2021-02-20T08:16:47.187 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Removing snap: 100% complete...done. 2021-02-20T08:16:47.214 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap4 2021-02-20T08:16:47.214 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_snapshot cluster2 mirror test snap4 2021-02-20T08:16:47.214 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:16:47.215 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:47.215 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:47.215 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap=snap4 2021-02-20T08:16:47.215 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap4 2021-02-20T08:16:48.170 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Removing snap: 100% complete...done. 2021-02-20T08:16:48.189 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap4-clean 2021-02-20T08:16:48.189 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_snapshot cluster2 mirror test snap4-clean 2021-02-20T08:16:48.189 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:16:48.190 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:48.190 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:48.190 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap=snap4-clean 2021-02-20T08:16:48.190 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap4-clean 2021-02-20T08:16:49.182 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Removing snap: 100% complete...done. 2021-02-20T08:16:49.198 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap5 2021-02-20T08:16:49.198 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_snapshot cluster2 mirror test snap5 2021-02-20T08:16:49.198 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:16:49.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:49.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:49.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap=snap5 2021-02-20T08:16:49.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap5 2021-02-20T08:16:50.187 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Removing snap: 100% complete...done. 2021-02-20T08:16:50.441 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap5-clean 2021-02-20T08:16:50.441 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_snapshot cluster2 mirror test snap5-clean 2021-02-20T08:16:50.442 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:16:50.442 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:50.442 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:50.443 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap=snap5-clean 2021-02-20T08:16:50.443 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap5-clean 2021-02-20T08:16:51.204 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Removing snap: 100% complete...done. 2021-02-20T08:16:51.220 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap6 2021-02-20T08:16:51.221 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_snapshot cluster2 mirror test snap6 2021-02-20T08:16:51.221 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:16:51.221 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:51.221 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:51.221 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap=snap6 2021-02-20T08:16:51.222 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap6 2021-02-20T08:16:52.204 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Removing snap: 100% complete...done. 2021-02-20T08:16:52.209 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap6-clean 2021-02-20T08:16:52.209 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_snapshot cluster2 mirror test snap6-clean 2021-02-20T08:16:52.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:16:52.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:52.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:52.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap=snap6-clean 2021-02-20T08:16:52.211 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap6-clean 2021-02-20T08:16:53.222 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Removing snap: 100% complete...done. 2021-02-20T08:16:53.259 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap7 2021-02-20T08:16:53.259 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_snapshot cluster2 mirror test snap7 2021-02-20T08:16:53.260 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:16:53.260 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:53.260 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:53.260 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap=snap7 2021-02-20T08:16:53.261 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap7 2021-02-20T08:16:54.082 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Removing snap: 100% complete...done. 2021-02-20T08:16:54.104 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap7-clean 2021-02-20T08:16:54.105 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_snapshot cluster2 mirror test snap7-clean 2021-02-20T08:16:54.105 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:16:54.105 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:54.105 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:54.106 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap=snap7-clean 2021-02-20T08:16:54.106 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap7-clean 2021-02-20T08:16:55.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Removing snap: 100% complete...done. 2021-02-20T08:16:55.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap8 2021-02-20T08:16:55.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_snapshot cluster2 mirror test snap8 2021-02-20T08:16:55.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:16:55.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:55.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:55.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap=snap8 2021-02-20T08:16:55.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap8 2021-02-20T08:16:56.220 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Removing snap: 100% complete...done. 2021-02-20T08:16:56.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap8-clean 2021-02-20T08:16:56.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_snapshot cluster2 mirror test snap8-clean 2021-02-20T08:16:56.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:16:56.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap=snap8-clean 2021-02-20T08:16:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap8-clean 2021-02-20T08:16:57.257 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Removing snap: 100% complete...done. 2021-02-20T08:16:57.288 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap9 2021-02-20T08:16:57.289 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_snapshot cluster2 mirror test snap9 2021-02-20T08:16:57.289 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:16:57.289 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:57.289 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:57.290 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap=snap9 2021-02-20T08:16:57.290 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap9 2021-02-20T08:16:58.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Removing snap: 100% complete...done. 2021-02-20T08:16:59.190 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap9-clean 2021-02-20T08:16:59.190 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_snapshot cluster2 mirror test snap9-clean 2021-02-20T08:16:59.191 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:16:59.191 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:16:59.191 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:16:59.191 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap=snap9-clean 2021-02-20T08:16:59.192 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap9-clean 2021-02-20T08:17:00.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Removing snap: 100% complete...done. 2021-02-20T08:17:00.290 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap10 2021-02-20T08:17:00.290 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_snapshot cluster2 mirror test snap10 2021-02-20T08:17:00.290 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:17:00.290 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:00.291 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:17:00.291 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap=snap10 2021-02-20T08:17:00.291 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap10 2021-02-20T08:17:01.260 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Removing snap: 100% complete...done. 2021-02-20T08:17:01.293 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap10-clean 2021-02-20T08:17:01.293 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_snapshot cluster2 mirror test snap10-clean 2021-02-20T08:17:01.293 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:17:01.294 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:01.294 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:17:01.294 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local snap=snap10-clean 2021-02-20T08:17:01.294 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap10-clean 2021-02-20T08:17:02.265 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Removing snap: 100% complete...done. 2021-02-20T08:17:02.294 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_image_retry cluster2 mirror test 2021-02-20T08:17:02.294 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:17:02.295 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:02.295 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:17:02.295 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0 2021-02-20T08:17:02.297 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ remove_image cluster2 mirror test 2021-02-20T08:17:02.297 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:17:02.298 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:02.298 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:17:02.298 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 snap purge mirror/test 2021-02-20T08:17:02.359 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 rm mirror/test 2021-02-20T08:17:03.809 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2021-02-20T08:17:03.813 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:17:03.813 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ wait_for_image_present cluster1 mirror test deleted 2021-02-20T08:17:03.814 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:17:03.814 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:03.814 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:17:03.814 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local state=deleted 2021-02-20T08:17:03.814 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image_id= 2021-02-20T08:17:03.815 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local s 2021-02-20T08:17:03.815 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -n 2021-02-20T08:17:03.815 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_image_id cluster1 mirror test 2021-02-20T08:17:03.815 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:17:03.815 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:03.816 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:17:03.816 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-02-20T08:17:03.816 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-02-20T08:17:03.888 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image_id=10a0aa806784 2021-02-20T08:17:03.889 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 0.1 2021-02-20T08:17:03.990 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_image_present cluster1 mirror test deleted 10a0aa806784 2021-02-20T08:17:03.991 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:17:03.991 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:03.991 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:17:03.991 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local test_state=deleted 2021-02-20T08:17:03.992 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image_id=10a0aa806784 2021-02-20T08:17:03.992 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_state=deleted 2021-02-20T08:17:03.992 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_image_id 2021-02-20T08:17:03.992 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_image_id cluster1 mirror test 2021-02-20T08:17:03.992 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:17:03.993 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:03.993 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:17:03.993 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-02-20T08:17:03.993 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-02-20T08:17:04.065 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_image_id=10a0aa806784 2021-02-20T08:17:04.066 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -n 10a0aa806784 2021-02-20T08:17:04.066 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -z 10a0aa806784 -o 10a0aa806784 = 10a0aa806784 2021-02-20T08:17:04.067 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_state=present 2021-02-20T08:17:04.067 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test deleted = present 2021-02-20T08:17:04.067 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 1 2021-02-20T08:17:05.067 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_image_present cluster1 mirror test deleted 10a0aa806784 2021-02-20T08:17:05.067 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:17:05.067 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:05.068 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:17:05.068 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local test_state=deleted 2021-02-20T08:17:05.068 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image_id=10a0aa806784 2021-02-20T08:17:05.068 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_state=deleted 2021-02-20T08:17:05.069 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_image_id 2021-02-20T08:17:05.069 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_image_id cluster1 mirror test 2021-02-20T08:17:05.069 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:17:05.069 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:05.069 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:17:05.070 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-02-20T08:17:05.070 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-02-20T08:17:05.148 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_image_id=10a0aa806784 2021-02-20T08:17:05.149 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -n 10a0aa806784 2021-02-20T08:17:05.149 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -z 10a0aa806784 -o 10a0aa806784 = 10a0aa806784 2021-02-20T08:17:05.149 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_state=present 2021-02-20T08:17:05.149 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test deleted = present 2021-02-20T08:17:05.150 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 2 2021-02-20T08:17:07.150 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_image_present cluster1 mirror test deleted 10a0aa806784 2021-02-20T08:17:07.150 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:17:07.151 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:07.151 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:17:07.151 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local test_state=deleted 2021-02-20T08:17:07.151 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image_id=10a0aa806784 2021-02-20T08:17:07.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_state=deleted 2021-02-20T08:17:07.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_image_id 2021-02-20T08:17:07.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_image_id cluster1 mirror test 2021-02-20T08:17:07.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:17:07.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:07.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:17:07.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-02-20T08:17:07.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-02-20T08:17:07.232 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_image_id=10a0aa806784 2021-02-20T08:17:07.232 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -n 10a0aa806784 2021-02-20T08:17:07.233 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -z 10a0aa806784 -o 10a0aa806784 = 10a0aa806784 2021-02-20T08:17:07.233 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_state=present 2021-02-20T08:17:07.234 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test deleted = present 2021-02-20T08:17:07.234 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 4 2021-02-20T08:17:11.234 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_image_present cluster1 mirror test deleted 10a0aa806784 2021-02-20T08:17:11.234 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:17:11.234 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:11.235 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:17:11.235 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local test_state=deleted 2021-02-20T08:17:11.235 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image_id=10a0aa806784 2021-02-20T08:17:11.235 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_state=deleted 2021-02-20T08:17:11.236 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_image_id 2021-02-20T08:17:11.236 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_image_id cluster1 mirror test 2021-02-20T08:17:11.236 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:17:11.236 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:11.236 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:17:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-02-20T08:17:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-02-20T08:17:11.310 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_image_id=10a0aa806784 2021-02-20T08:17:11.310 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -n 10a0aa806784 2021-02-20T08:17:11.310 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -z 10a0aa806784 -o 10a0aa806784 = 10a0aa806784 2021-02-20T08:17:11.311 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_state=present 2021-02-20T08:17:11.311 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test deleted = present 2021-02-20T08:17:11.311 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 8 2021-02-20T08:17:19.312 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_image_present cluster1 mirror test deleted 10a0aa806784 2021-02-20T08:17:19.312 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:17:19.312 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:19.313 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:17:19.313 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local test_state=deleted 2021-02-20T08:17:19.313 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image_id=10a0aa806784 2021-02-20T08:17:19.313 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_state=deleted 2021-02-20T08:17:19.314 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_image_id 2021-02-20T08:17:19.314 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_image_id cluster1 mirror test 2021-02-20T08:17:19.314 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:17:19.314 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:19.314 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:17:19.315 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-02-20T08:17:19.315 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-02-20T08:17:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_image_id=10a0aa806784 2021-02-20T08:17:19.389 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -n 10a0aa806784 2021-02-20T08:17:19.389 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -z 10a0aa806784 -o 10a0aa806784 = 10a0aa806784 2021-02-20T08:17:19.390 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_state=present 2021-02-20T08:17:19.390 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test deleted = present 2021-02-20T08:17:19.390 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sleep 8 2021-02-20T08:17:27.391 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test_image_present cluster1 mirror test deleted 10a0aa806784 2021-02-20T08:17:27.392 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:17:27.392 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:27.392 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:17:27.393 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local test_state=deleted 2021-02-20T08:17:27.393 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image_id=10a0aa806784 2021-02-20T08:17:27.393 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_state=deleted 2021-02-20T08:17:27.393 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local current_image_id 2021-02-20T08:17:27.393 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ get_image_id cluster1 mirror test 2021-02-20T08:17:27.394 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster1 2021-02-20T08:17:27.394 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:27.394 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=test 2021-02-20T08:17:27.395 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-02-20T08:17:27.395 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-02-20T08:17:27.435 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:rbd: error opening image test: (2) No such file or directory 2021-02-20T08:17:27.440 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ current_image_id= 2021-02-20T08:17:27.441 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test -n 2021-02-20T08:17:27.441 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ test deleted = deleted 2021-02-20T08:17:27.441 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:17:27.442 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ testlog TEST: create many images 2021-02-20T08:17:27.442 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ tee -a /tmp/tmp.1RV4oXZGap/rbd-mirror.test.log 2021-02-20T08:17:27.442 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ date +%F %T 2021-02-20T08:17:27.444 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ echo 2021-02-20 08:17:27 TEST: create many images 2021-02-20T08:17:27.444 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:2021-02-20 08:17:27 TEST: create many images 2021-02-20T08:17:27.444 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ snap_name=snap 2021-02-20T08:17:27.445 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ seq 1 50 2021-02-20T08:17:27.445 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_1 2021-02-20T08:17:27.445 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_1 snapshot 128M 2021-02-20T08:17:27.445 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:17:27.446 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:27.446 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:27.447 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:27.447 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_1 2021-02-20T08:17:27.447 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:27.447 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:17:27.448 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:17:27.448 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:27.448 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_1 128M 2021-02-20T08:17:27.448 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:17:27.449 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:27.449 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:27.449 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:27.450 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_1 2021-02-20T08:17:27.450 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:27.450 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:17:27.451 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:17:27.451 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:17:27.451 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:27.452 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_1 2021-02-20T08:17:27.527 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:17:27.527 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_1 snapshot 2021-02-20T08:17:27.527 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:17:27.527 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:27.528 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_1 2021-02-20T08:17:27.528 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:17:27.528 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_1 snapshot 2021-02-20T08:17:28.160 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:17:28.194 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:17:28.195 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_1 2021-02-20T08:17:28.195 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:17:28.195 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:28.196 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_1 2021-02-20T08:17:28.196 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:17:28.197 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=7 2021-02-20T08:17:28.197 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:17:28.197 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_1 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:17:28.658 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 21226496 bytes 2021-02-20T08:17:28.871 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 51220480 bytes 2021-02-20T08:17:29.043 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 58256384 bytes 2021-02-20T08:17:29.397 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 158848000 bytes 2021-02-20T08:17:29.641 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 199086080 bytes 2021-02-20T08:17:29.889 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 247237632 bytes 2021-02-20T08:17:30.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 266363904 bytes 2021-02-20T08:17:30.284 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 278300672 bytes 2021-02-20T08:17:30.566 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 339107840 bytes 2021-02-20T08:17:30.844 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 387171328 bytes 2021-02-20T08:17:31.131 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 453875712 bytes 2021-02-20T08:17:31.352 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 497004544 bytes 2021-02-20T08:17:31.609 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 525047808 bytes 2021-02-20T08:17:31.828 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 557325312 bytes 2021-02-20T08:17:32.018 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 576925696 bytes 2021-02-20T08:17:32.333 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 670743552 bytes 2021-02-20T08:17:32.705 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 748667904 bytes 2021-02-20T08:17:32.861 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 749290496 bytes 2021-02-20T08:17:33.084 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 778182656 bytes 2021-02-20T08:17:33.298 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 808467456 bytes 2021-02-20T08:17:33.459 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 821859328 bytes 2021-02-20T08:17:33.753 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 871803904 bytes 2021-02-20T08:17:33.998 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 907154432 bytes 2021-02-20T08:17:34.218 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 928817152 bytes 2021-02-20T08:17:34.254 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8269086590345586) > weight (0.25) 2021-02-20T08:17:34.254 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7426809462376216) > weight (0.25) 2021-02-20T08:17:34.255 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9943626849761434) > weight (0.25) 2021-02-20T08:17:34.255 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T08:17:34.256 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T08:17:34.256 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T08:17:34.257 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T08:17:34.251+0000 7fe388e97700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 25093) UID: 1000 2021-02-20T08:17:34.514 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 989006848 bytes 2021-02-20T08:17:34.658 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:17:34.820 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 1059784704 bytes 2021-02-20T08:17:35.072 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2700 IOs, 1089043456 bytes 2021-02-20T08:17:35.203 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:17:35.204 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:17:35.204 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:17:35.205 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:17:35.205 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_2 2021-02-20T08:17:35.206 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_2 snapshot 128M 2021-02-20T08:17:35.206 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:17:35.206 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:35.207 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:35.207 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:35.208 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_2 2021-02-20T08:17:35.208 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:35.208 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:17:35.209 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:17:35.209 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:35.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_2 128M 2021-02-20T08:17:35.213 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:17:35.214 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:35.214 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:35.215 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:35.215 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_2 2021-02-20T08:17:35.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:35.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:17:35.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:17:35.217 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:17:35.217 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:35.218 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_2 2021-02-20T08:17:35.274 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:17:35.275 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_2 snapshot 2021-02-20T08:17:35.275 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:17:35.275 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:35.275 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_2 2021-02-20T08:17:35.276 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:17:35.276 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_2 snapshot 2021-02-20T08:17:36.163 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:17:36.190 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:17:36.190 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_2 2021-02-20T08:17:36.191 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:17:36.191 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:36.191 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_2 2021-02-20T08:17:36.191 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:17:36.192 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=9 2021-02-20T08:17:36.192 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:17:36.193 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_2 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:17:36.664 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 21399552 bytes 2021-02-20T08:17:36.848 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 38062080 bytes 2021-02-20T08:17:37.018 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 48494592 bytes 2021-02-20T08:17:37.419 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 165979136 bytes 2021-02-20T08:17:37.637 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 204820480 bytes 2021-02-20T08:17:37.972 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 277927936 bytes 2021-02-20T08:17:38.184 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 297453568 bytes 2021-02-20T08:17:38.386 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 318264320 bytes 2021-02-20T08:17:38.675 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 373277696 bytes 2021-02-20T08:17:38.880 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 395055104 bytes 2021-02-20T08:17:39.137 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 438625280 bytes 2021-02-20T08:17:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 476571648 bytes 2021-02-20T08:17:39.635 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 528452608 bytes 2021-02-20T08:17:39.799 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 535041024 bytes 2021-02-20T08:17:40.036 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 567185408 bytes 2021-02-20T08:17:40.257 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T08:17:40.257 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:17:40.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:17:40.258 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T08:17:40.258 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T08:17:40.259 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T08:17:40.261 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T08:17:40.262 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T08:17:40.281 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 609302528 bytes 2021-02-20T08:17:40.552 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 658632704 bytes 2021-02-20T08:17:40.751 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 671704064 bytes 2021-02-20T08:17:40.997 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 708372480 bytes 2021-02-20T08:17:41.209 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 731448320 bytes 2021-02-20T08:17:41.649 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 860980224 bytes 2021-02-20T08:17:41.882 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 886506496 bytes 2021-02-20T08:17:42.142 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 928571392 bytes 2021-02-20T08:17:42.472 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 1017353216 bytes 2021-02-20T08:17:42.659 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 1027364864 bytes 2021-02-20T08:17:42.929 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 1078797312 bytes 2021-02-20T08:17:43.270 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2700 IOs, 1162393600 bytes 2021-02-20T08:17:43.556 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2800 IOs, 1232093184 bytes 2021-02-20T08:17:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2900 IOs, 1257239552 bytes 2021-02-20T08:17:44.144 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3000 IOs, 1259024384 bytes 2021-02-20T08:17:44.458 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3100 IOs, 1327727616 bytes 2021-02-20T08:17:44.760 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3200 IOs, 1392953344 bytes 2021-02-20T08:17:44.999 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3300 IOs, 1426410496 bytes 2021-02-20T08:17:45.197 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:17:45.197 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:17:45.198 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:17:45.198 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:17:45.198 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_3 2021-02-20T08:17:45.198 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_3 snapshot 128M 2021-02-20T08:17:45.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:17:45.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:45.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:45.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:45.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_3 2021-02-20T08:17:45.200 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:45.200 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:17:45.200 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:17:45.200 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:45.200 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_3 128M 2021-02-20T08:17:45.201 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:17:45.201 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:45.201 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:45.201 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:45.201 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_3 2021-02-20T08:17:45.202 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:45.202 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:17:45.202 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:17:45.202 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:17:45.203 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:45.203 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_3 2021-02-20T08:17:45.279 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:17:45.279 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_3 snapshot 2021-02-20T08:17:45.280 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:17:45.280 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:45.280 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_3 2021-02-20T08:17:45.280 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:17:45.281 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_3 snapshot 2021-02-20T08:17:46.082 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6187816103691243) > weight (0.25) 2021-02-20T08:17:46.083 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T08:17:46.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T08:17:46.084 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T08:17:46.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T08:17:46.079+0000 7f3cb41ea700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 18977) UID: 1000 2021-02-20T08:17:46.174 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:17:46.200 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:17:46.200 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_3 2021-02-20T08:17:46.201 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:17:46.201 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:46.201 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_3 2021-02-20T08:17:46.201 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:17:46.202 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=11 2021-02-20T08:17:46.202 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:17:46.203 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_3 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:17:46.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:17:46.704 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 27504640 bytes 2021-02-20T08:17:46.891 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 43166720 bytes 2021-02-20T08:17:47.133 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 95195136 bytes 2021-02-20T08:17:47.326 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 116139008 bytes 2021-02-20T08:17:47.598 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 151745536 bytes 2021-02-20T08:17:47.981 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 255875072 bytes 2021-02-20T08:17:48.155 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 256507904 bytes 2021-02-20T08:17:48.471 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 322086912 bytes 2021-02-20T08:17:48.875 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 398838784 bytes 2021-02-20T08:17:49.074 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 400991232 bytes 2021-02-20T08:17:49.086 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T08:17:49.086 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:17:49.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:17:49.087 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T08:17:49.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T08:17:49.087 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T08:17:49.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T08:17:49.090 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T08:17:49.349 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 449755136 bytes 2021-02-20T08:17:49.537 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 455407616 bytes 2021-02-20T08:17:49.763 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 496918528 bytes 2021-02-20T08:17:50.057 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 552951808 bytes 2021-02-20T08:17:50.415 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 618677248 bytes 2021-02-20T08:17:50.823 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 752024576 bytes 2021-02-20T08:17:51.092 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 781557760 bytes 2021-02-20T08:17:51.289 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 794646528 bytes 2021-02-20T08:17:51.504 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 829264896 bytes 2021-02-20T08:17:51.716 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 832553984 bytes 2021-02-20T08:17:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 926951424 bytes 2021-02-20T08:17:52.300 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 954632192 bytes 2021-02-20T08:17:52.576 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 1023436800 bytes 2021-02-20T08:17:52.928 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 1096265728 bytes 2021-02-20T08:17:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 1102260224 bytes 2021-02-20T08:17:53.344 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 1142039552 bytes 2021-02-20T08:17:53.627 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2700 IOs, 1209199616 bytes 2021-02-20T08:17:54.046 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2800 IOs, 1227940864 bytes 2021-02-20T08:17:54.378 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2900 IOs, 1307700224 bytes 2021-02-20T08:17:54.632 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3000 IOs, 1341745152 bytes 2021-02-20T08:17:55.036 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3100 IOs, 1452797952 bytes 2021-02-20T08:17:55.243 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3200 IOs, 1478350848 bytes 2021-02-20T08:17:55.583 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3300 IOs, 1563652096 bytes 2021-02-20T08:17:55.826 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3400 IOs, 1619087360 bytes 2021-02-20T08:17:56.064 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3500 IOs, 1640867840 bytes 2021-02-20T08:17:56.378 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3600 IOs, 1706680320 bytes 2021-02-20T08:17:56.627 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3700 IOs, 1744801792 bytes 2021-02-20T08:17:56.912 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3800 IOs, 1843696640 bytes 2021-02-20T08:17:57.209 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:17:57.209 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:17:57.209 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:17:57.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:17:57.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_4 2021-02-20T08:17:57.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_4 snapshot 128M 2021-02-20T08:17:57.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:17:57.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:57.211 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:57.211 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:57.211 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_4 2021-02-20T08:17:57.211 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:57.211 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:17:57.212 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:17:57.212 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:57.212 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_4 128M 2021-02-20T08:17:57.212 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:17:57.212 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:57.213 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:57.213 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:57.213 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_4 2021-02-20T08:17:57.213 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:57.214 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:17:57.214 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:17:57.214 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:17:57.215 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:17:57.215 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_4 2021-02-20T08:17:57.286 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:17:57.286 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_4 snapshot 2021-02-20T08:17:57.286 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:17:57.286 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:57.287 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_4 2021-02-20T08:17:57.287 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:17:57.287 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_4 snapshot 2021-02-20T08:17:58.166 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:17:58.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:17:58.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_4 2021-02-20T08:17:58.199 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:17:58.200 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:17:58.200 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_4 2021-02-20T08:17:58.200 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:17:58.201 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=10 2021-02-20T08:17:58.201 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:17:58.201 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_4 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:17:58.728 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 46358528 bytes 2021-02-20T08:17:59.151 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 83735552 bytes 2021-02-20T08:17:59.577 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 136739840 bytes 2021-02-20T08:17:59.766 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 148531200 bytes 2021-02-20T08:18:00.007 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 181950464 bytes 2021-02-20T08:18:00.186 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 191754240 bytes 2021-02-20T08:18:00.432 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 225431552 bytes 2021-02-20T08:18:00.696 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 269406208 bytes 2021-02-20T08:18:00.903 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 291673088 bytes 2021-02-20T08:18:01.204 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 354361344 bytes 2021-02-20T08:18:01.557 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 442126336 bytes 2021-02-20T08:18:01.833 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 496406528 bytes 2021-02-20T08:18:02.070 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 533268480 bytes 2021-02-20T08:18:02.278 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 550437888 bytes 2021-02-20T08:18:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 580788224 bytes 2021-02-20T08:18:02.797 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 626719744 bytes 2021-02-20T08:18:03.087 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 683056128 bytes 2021-02-20T08:18:03.297 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 702243840 bytes 2021-02-20T08:18:03.623 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 802947072 bytes 2021-02-20T08:18:03.910 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 861139968 bytes 2021-02-20T08:18:04.577 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 926487552 bytes 2021-02-20T08:18:04.879 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 980134912 bytes 2021-02-20T08:18:05.126 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 1018835968 bytes 2021-02-20T08:18:05.294 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 1019439104 bytes 2021-02-20T08:18:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 1068125184 bytes 2021-02-20T08:18:05.833 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 1117999104 bytes 2021-02-20T08:18:06.135 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2700 IOs, 1205311488 bytes 2021-02-20T08:18:06.486 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2800 IOs, 1272849408 bytes 2021-02-20T08:18:06.781 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2900 IOs, 1334517760 bytes 2021-02-20T08:18:07.041 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3000 IOs, 1372197888 bytes 2021-02-20T08:18:07.365 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3100 IOs, 1444541440 bytes 2021-02-20T08:18:07.739 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3200 IOs, 1539474432 bytes 2021-02-20T08:18:07.928 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3300 IOs, 1556329472 bytes 2021-02-20T08:18:08.205 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:18:08.206 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:18:08.206 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:18:08.206 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:18:08.206 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_5 2021-02-20T08:18:08.207 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_5 snapshot 128M 2021-02-20T08:18:08.207 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:08.207 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:08.207 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:08.208 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:08.208 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_5 2021-02-20T08:18:08.208 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:08.208 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:18:08.208 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:18:08.209 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:08.209 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_5 128M 2021-02-20T08:18:08.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:08.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:08.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:08.210 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:08.211 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_5 2021-02-20T08:18:08.211 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:08.211 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:18:08.211 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:18:08.212 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:18:08.212 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:08.212 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_5 2021-02-20T08:18:08.395 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:18:08.395 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_5 snapshot 2021-02-20T08:18:08.395 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:08.396 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:08.396 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_5 2021-02-20T08:18:08.396 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:18:08.396 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_5 snapshot 2021-02-20T08:18:09.443 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:18:09.619 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:18:09.620 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_5 2021-02-20T08:18:09.620 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:09.620 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:09.620 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_5 2021-02-20T08:18:09.621 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:18:09.814 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=5 2021-02-20T08:18:09.814 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:18:09.815 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_5 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:18:10.418 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 78905344 bytes 2021-02-20T08:18:10.657 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 94818304 bytes 2021-02-20T08:18:10.835 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 101210112 bytes 2021-02-20T08:18:11.121 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 156994560 bytes 2021-02-20T08:18:11.339 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 197856256 bytes 2021-02-20T08:18:11.648 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 259450880 bytes 2021-02-20T08:18:11.931 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 303767552 bytes 2021-02-20T08:18:12.123 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 335086592 bytes 2021-02-20T08:18:12.376 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 364127232 bytes 2021-02-20T08:18:12.657 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 415224832 bytes 2021-02-20T08:18:12.946 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 477733888 bytes 2021-02-20T08:18:13.188 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 507789312 bytes 2021-02-20T08:18:13.467 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 550118400 bytes 2021-02-20T08:18:13.634 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 550712320 bytes 2021-02-20T08:18:13.978 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 632261632 bytes 2021-02-20T08:18:14.364 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 731606016 bytes 2021-02-20T08:18:14.883 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 763815936 bytes 2021-02-20T08:18:15.106 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:18:15.107 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:18:15.107 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:18:15.107 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:18:15.107 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_6 2021-02-20T08:18:15.108 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_6 snapshot 128M 2021-02-20T08:18:15.108 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:15.108 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:15.108 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:15.108 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:15.109 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_6 2021-02-20T08:18:15.109 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:15.109 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:18:15.109 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:18:15.109 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:15.110 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_6 128M 2021-02-20T08:18:15.110 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:15.110 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:15.110 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:15.110 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:15.111 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_6 2021-02-20T08:18:15.111 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:15.111 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:18:15.111 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:18:15.112 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:18:15.112 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:15.112 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_6 2021-02-20T08:18:15.332 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:18:15.333 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_6 snapshot 2021-02-20T08:18:15.333 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:15.333 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:15.334 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_6 2021-02-20T08:18:15.334 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:18:15.334 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_6 snapshot 2021-02-20T08:18:16.195 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:18:16.232 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:18:16.232 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_6 2021-02-20T08:18:16.232 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:16.233 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:16.233 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_6 2021-02-20T08:18:16.233 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:18:16.379 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=6 2021-02-20T08:18:16.380 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:18:16.380 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_6 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:18:16.884 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 17103872 bytes 2021-02-20T08:18:17.194 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 94327808 bytes 2021-02-20T08:18:17.434 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 129726464 bytes 2021-02-20T08:18:17.630 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 143612928 bytes 2021-02-20T08:18:17.828 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 157708288 bytes 2021-02-20T08:18:18.094 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 210828288 bytes 2021-02-20T08:18:18.293 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 223796224 bytes 2021-02-20T08:18:18.623 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 318167040 bytes 2021-02-20T08:18:18.951 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 370489344 bytes 2021-02-20T08:18:19.185 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 398576640 bytes 2021-02-20T08:18:19.424 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 437527552 bytes 2021-02-20T08:18:19.634 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 460430336 bytes 2021-02-20T08:18:20.159 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 508327936 bytes 2021-02-20T08:18:20.427 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 559301632 bytes 2021-02-20T08:18:20.802 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 651494400 bytes 2021-02-20T08:18:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 675649536 bytes 2021-02-20T08:18:21.346 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 760908800 bytes 2021-02-20T08:18:21.601 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 784376832 bytes 2021-02-20T08:18:21.819 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 811616256 bytes 2021-02-20T08:18:22.018 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 820746240 bytes 2021-02-20T08:18:22.202 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 841126912 bytes 2021-02-20T08:18:22.387 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:18:22.388 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:18:22.388 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:18:22.388 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:18:22.389 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_7 2021-02-20T08:18:22.389 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_7 snapshot 128M 2021-02-20T08:18:22.389 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:22.389 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:22.389 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:22.390 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:22.390 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_7 2021-02-20T08:18:22.390 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:22.391 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:18:22.391 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:18:22.391 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:22.391 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_7 128M 2021-02-20T08:18:22.392 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:22.392 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:22.392 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:22.393 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:22.393 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_7 2021-02-20T08:18:22.393 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:22.394 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:18:22.394 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:18:22.394 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:18:22.395 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:22.395 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_7 2021-02-20T08:18:22.469 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:18:22.469 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_7 snapshot 2021-02-20T08:18:22.469 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:22.470 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:22.470 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_7 2021-02-20T08:18:22.470 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:18:22.470 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_7 snapshot 2021-02-20T08:18:23.211 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:18:23.244 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:18:23.244 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_7 2021-02-20T08:18:23.244 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:23.245 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:23.245 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_7 2021-02-20T08:18:23.245 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:18:23.246 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=7 2021-02-20T08:18:23.246 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:18:23.247 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_7 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:18:23.930 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 1040384 bytes 2021-02-20T08:18:24.173 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 40297472 bytes 2021-02-20T08:18:24.397 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 68772864 bytes 2021-02-20T08:18:24.626 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 99447808 bytes 2021-02-20T08:18:25.016 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 135306240 bytes 2021-02-20T08:18:25.273 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 187491328 bytes 2021-02-20T08:18:25.544 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 213129216 bytes 2021-02-20T08:18:25.823 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 260901888 bytes 2021-02-20T08:18:26.151 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 325821440 bytes 2021-02-20T08:18:26.465 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 385909760 bytes 2021-02-20T08:18:26.679 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 405786624 bytes 2021-02-20T08:18:26.943 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 457403392 bytes 2021-02-20T08:18:27.175 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 491681792 bytes 2021-02-20T08:18:27.560 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 587539456 bytes 2021-02-20T08:18:27.810 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 619411456 bytes 2021-02-20T08:18:28.064 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 655943680 bytes 2021-02-20T08:18:28.352 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 711681024 bytes 2021-02-20T08:18:28.548 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 744722432 bytes 2021-02-20T08:18:28.888 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 807841792 bytes 2021-02-20T08:18:29.138 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 855180288 bytes 2021-02-20T08:18:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 887380992 bytes 2021-02-20T08:18:29.920 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 1002899456 bytes 2021-02-20T08:18:30.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 1053813760 bytes 2021-02-20T08:18:30.250 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:18:30.250 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:18:30.250 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:18:30.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:18:30.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_8 2021-02-20T08:18:30.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_8 snapshot 128M 2021-02-20T08:18:30.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:30.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:30.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:30.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:30.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_8 2021-02-20T08:18:30.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:30.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:18:30.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:18:30.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:30.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_8 128M 2021-02-20T08:18:30.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:30.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:30.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:30.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:30.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_8 2021-02-20T08:18:30.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:30.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:18:30.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:18:30.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:18:30.256 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:30.256 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_8 2021-02-20T08:18:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:18:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_8 snapshot 2021-02-20T08:18:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_8 2021-02-20T08:18:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:18:30.608 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_8 snapshot 2021-02-20T08:18:31.192 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:18:31.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:18:31.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_8 2021-02-20T08:18:31.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:31.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:31.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_8 2021-02-20T08:18:31.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:18:31.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=10 2021-02-20T08:18:31.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:18:31.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_8 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:18:31.747 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 39985152 bytes 2021-02-20T08:18:31.971 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 76880896 bytes 2021-02-20T08:18:32.266 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 118899712 bytes 2021-02-20T08:18:32.504 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 155267072 bytes 2021-02-20T08:18:32.667 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 155884544 bytes 2021-02-20T08:18:32.927 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 199197696 bytes 2021-02-20T08:18:33.155 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 232648704 bytes 2021-02-20T08:18:33.558 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 328964096 bytes 2021-02-20T08:18:33.766 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 342437888 bytes 2021-02-20T08:18:34.127 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 421914624 bytes 2021-02-20T08:18:34.440 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 485702656 bytes 2021-02-20T08:18:34.688 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 517176320 bytes 2021-02-20T08:18:35.147 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 610294784 bytes 2021-02-20T08:18:35.324 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 610932736 bytes 2021-02-20T08:18:35.659 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 685176832 bytes 2021-02-20T08:18:35.912 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 713953280 bytes 2021-02-20T08:18:36.294 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 813390848 bytes 2021-02-20T08:18:36.605 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 876990464 bytes 2021-02-20T08:18:36.814 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 896798720 bytes 2021-02-20T08:18:37.126 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 962903040 bytes 2021-02-20T08:18:37.391 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 1008664576 bytes 2021-02-20T08:18:37.603 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 1025905664 bytes 2021-02-20T08:18:37.890 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 1074095104 bytes 2021-02-20T08:18:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 1119677440 bytes 2021-02-20T08:18:38.516 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 1198234624 bytes 2021-02-20T08:18:38.934 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 1311304704 bytes 2021-02-20T08:18:39.153 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2700 IOs, 1332190208 bytes 2021-02-20T08:18:39.405 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2800 IOs, 1369374720 bytes 2021-02-20T08:18:39.620 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2900 IOs, 1394859008 bytes 2021-02-20T08:18:40.123 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3000 IOs, 1452688384 bytes 2021-02-20T08:18:40.416 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3100 IOs, 1487253504 bytes 2021-02-20T08:18:40.674 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3200 IOs, 1527600128 bytes 2021-02-20T08:18:40.873 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3300 IOs, 1542833152 bytes 2021-02-20T08:18:41.104 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3400 IOs, 1573133312 bytes 2021-02-20T08:18:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:18:41.232 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:18:41.232 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:18:41.232 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:18:41.232 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_9 2021-02-20T08:18:41.233 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_9 snapshot 128M 2021-02-20T08:18:41.233 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:41.233 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:41.233 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:41.233 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:41.234 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_9 2021-02-20T08:18:41.234 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:41.234 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:18:41.234 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:18:41.235 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:41.235 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_9 128M 2021-02-20T08:18:41.235 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:41.235 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:41.235 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_9 2021-02-20T08:18:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:18:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:18:41.237 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:18:41.238 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:41.238 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_9 2021-02-20T08:18:41.600 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:18:41.600 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_9 snapshot 2021-02-20T08:18:41.601 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:41.601 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:41.601 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_9 2021-02-20T08:18:41.601 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:18:41.602 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_9 snapshot 2021-02-20T08:18:42.184 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:18:42.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:18:42.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_9 2021-02-20T08:18:42.217 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:42.217 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:42.217 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_9 2021-02-20T08:18:42.217 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:18:42.218 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=13 2021-02-20T08:18:42.218 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:18:42.218 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_9 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:18:42.733 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 45777920 bytes 2021-02-20T08:18:43.100 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 144253952 bytes 2021-02-20T08:18:43.304 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 157040640 bytes 2021-02-20T08:18:43.566 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 192932864 bytes 2021-02-20T08:18:43.813 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 222680064 bytes 2021-02-20T08:18:44.127 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 294670336 bytes 2021-02-20T08:18:44.542 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 421715968 bytes 2021-02-20T08:18:44.870 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 482288640 bytes 2021-02-20T08:18:45.069 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 482887680 bytes 2021-02-20T08:18:45.572 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 546653184 bytes 2021-02-20T08:18:45.836 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 585209856 bytes 2021-02-20T08:18:46.084 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 614233088 bytes 2021-02-20T08:18:46.282 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 620204032 bytes 2021-02-20T08:18:46.573 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 680134656 bytes 2021-02-20T08:18:46.774 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 697031680 bytes 2021-02-20T08:18:47.261 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 773590016 bytes 2021-02-20T08:18:47.567 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 847809536 bytes 2021-02-20T08:18:47.842 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 888931328 bytes 2021-02-20T08:18:48.165 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 956308480 bytes 2021-02-20T08:18:48.395 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 988120064 bytes 2021-02-20T08:18:48.804 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 1030247424 bytes 2021-02-20T08:18:49.116 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 1083980800 bytes 2021-02-20T08:18:49.336 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 1106049024 bytes 2021-02-20T08:18:49.581 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 1136227328 bytes 2021-02-20T08:18:49.823 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 1164883968 bytes 2021-02-20T08:18:50.075 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 1203355648 bytes 2021-02-20T08:18:50.569 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2700 IOs, 1251442688 bytes 2021-02-20T08:18:50.902 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2800 IOs, 1334231040 bytes 2021-02-20T08:18:51.139 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2900 IOs, 1355230208 bytes 2021-02-20T08:18:51.384 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3000 IOs, 1396015104 bytes 2021-02-20T08:18:51.597 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3100 IOs, 1416099840 bytes 2021-02-20T08:18:51.912 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3200 IOs, 1489548288 bytes 2021-02-20T08:18:52.279 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3300 IOs, 1569168384 bytes 2021-02-20T08:18:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3400 IOs, 1608636416 bytes 2021-02-20T08:18:52.871 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3500 IOs, 1691941888 bytes 2021-02-20T08:18:53.337 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3600 IOs, 1722735616 bytes 2021-02-20T08:18:53.658 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3700 IOs, 1785616384 bytes 2021-02-20T08:18:53.901 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3800 IOs, 1818934272 bytes 2021-02-20T08:18:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3900 IOs, 1855776768 bytes 2021-02-20T08:18:54.428 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4000 IOs, 1885136896 bytes 2021-02-20T08:18:54.697 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4100 IOs, 1935046656 bytes 2021-02-20T08:18:55.067 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4200 IOs, 2039785472 bytes 2021-02-20T08:18:55.223 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:18:55.223 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:18:55.223 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:18:55.223 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:18:55.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_10 2021-02-20T08:18:55.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_10 snapshot 128M 2021-02-20T08:18:55.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:55.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:55.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:55.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:55.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_10 2021-02-20T08:18:55.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:55.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:18:55.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:18:55.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:55.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_10 128M 2021-02-20T08:18:55.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:55.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:55.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:55.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:55.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_10 2021-02-20T08:18:55.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:55.228 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:18:55.228 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:18:55.228 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:18:55.228 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:18:55.229 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_10 2021-02-20T08:18:55.412 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:18:55.412 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_10 snapshot 2021-02-20T08:18:55.412 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:55.413 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:55.413 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_10 2021-02-20T08:18:55.413 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:18:55.413 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_10 snapshot 2021-02-20T08:18:56.178 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:18:56.215 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:18:56.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_10 2021-02-20T08:18:56.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:18:56.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:18:56.216 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_10 2021-02-20T08:18:56.217 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:18:56.217 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=12 2021-02-20T08:18:56.218 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:18:56.218 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_10 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:18:56.661 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 17132544 bytes 2021-02-20T08:18:56.946 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 70938624 bytes 2021-02-20T08:18:57.168 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 101984256 bytes 2021-02-20T08:18:57.402 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 136420352 bytes 2021-02-20T08:18:57.620 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 164434944 bytes 2021-02-20T08:18:57.930 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 276762624 bytes 2021-02-20T08:18:58.310 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 326531072 bytes 2021-02-20T08:18:58.589 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 388319232 bytes 2021-02-20T08:18:58.832 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 401689600 bytes 2021-02-20T08:18:59.043 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 419116032 bytes 2021-02-20T08:18:59.308 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 481719296 bytes 2021-02-20T08:18:59.644 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 553751552 bytes 2021-02-20T08:18:59.961 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 620359680 bytes 2021-02-20T08:19:00.189 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 654343168 bytes 2021-02-20T08:19:00.428 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 692927488 bytes 2021-02-20T08:19:00.829 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 754522112 bytes 2021-02-20T08:19:01.005 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 756316160 bytes 2021-02-20T08:19:01.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 785473536 bytes 2021-02-20T08:19:01.545 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 843503616 bytes 2021-02-20T08:19:01.785 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 877233152 bytes 2021-02-20T08:19:02.008 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 912927744 bytes 2021-02-20T08:19:02.277 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 951061504 bytes 2021-02-20T08:19:02.536 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 989863936 bytes 2021-02-20T08:19:02.818 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 1043139584 bytes 2021-02-20T08:19:03.099 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 1099934720 bytes 2021-02-20T08:19:03.298 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 1111526400 bytes 2021-02-20T08:19:03.712 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2700 IOs, 1229368320 bytes 2021-02-20T08:19:03.924 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2800 IOs, 1243059200 bytes 2021-02-20T08:19:04.175 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2900 IOs, 1282177024 bytes 2021-02-20T08:19:04.392 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3000 IOs, 1304636416 bytes 2021-02-20T08:19:04.646 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3100 IOs, 1360065536 bytes 2021-02-20T08:19:05.005 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3200 IOs, 1426301952 bytes 2021-02-20T08:19:05.301 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3300 IOs, 1482245120 bytes 2021-02-20T08:19:06.096 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3400 IOs, 1536587776 bytes 2021-02-20T08:19:06.553 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3500 IOs, 1656514560 bytes 2021-02-20T08:19:06.895 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3600 IOs, 1737224192 bytes 2021-02-20T08:19:07.262 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3700 IOs, 1818612736 bytes 2021-02-20T08:19:07.565 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3800 IOs, 1890846720 bytes 2021-02-20T08:19:07.980 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3900 IOs, 1988946944 bytes 2021-02-20T08:19:08.223 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:19:08.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:19:08.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:19:08.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:19:08.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_11 2021-02-20T08:19:08.224 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_11 snapshot 128M 2021-02-20T08:19:08.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:19:08.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:08.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:19:08.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:08.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_11 2021-02-20T08:19:08.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:08.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:19:08.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:19:08.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:08.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_11 128M 2021-02-20T08:19:08.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:19:08.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:08.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:19:08.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:08.228 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_11 2021-02-20T08:19:08.228 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:08.228 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:19:08.228 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:19:08.228 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:19:08.229 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:08.229 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_11 2021-02-20T08:19:08.542 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:19:08.543 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_11 snapshot 2021-02-20T08:19:08.543 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:19:08.543 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:19:08.543 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_11 2021-02-20T08:19:08.543 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:19:08.544 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_11 snapshot 2021-02-20T08:19:09.208 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:19:09.238 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:19:09.238 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_11 2021-02-20T08:19:09.238 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:19:09.238 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:19:09.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_11 2021-02-20T08:19:09.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:19:09.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=11 2021-02-20T08:19:09.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:19:09.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_11 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:19:09.776 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 20063232 bytes 2021-02-20T08:19:10.009 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 55699456 bytes 2021-02-20T08:19:10.278 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 105284608 bytes 2021-02-20T08:19:10.523 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 146257920 bytes 2021-02-20T08:19:10.734 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 166332416 bytes 2021-02-20T08:19:10.919 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 172411904 bytes 2021-02-20T08:19:11.103 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 183350272 bytes 2021-02-20T08:19:11.348 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 213633024 bytes 2021-02-20T08:19:11.645 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 263194624 bytes 2021-02-20T08:19:11.867 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 309775360 bytes 2021-02-20T08:19:12.091 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.49776963329293256) > weight (0.25) 2021-02-20T08:19:12.092 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T08:19:12.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T08:19:12.093 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T08:19:12.093 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T08:19:12.093 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:19:12.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T08:19:12.086+0000 7feb67df0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 28049) UID: 1000 2021-02-20T08:19:12.213 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 371466240 bytes 2021-02-20T08:19:12.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:19:12.581 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 480680960 bytes 2021-02-20T08:19:12.869 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 530107392 bytes 2021-02-20T08:19:13.198 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 615437312 bytes 2021-02-20T08:19:13.418 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 636967936 bytes 2021-02-20T08:19:14.082 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 666824704 bytes 2021-02-20T08:19:14.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 692105216 bytes 2021-02-20T08:19:14.497 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 702309376 bytes 2021-02-20T08:19:14.812 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 769375232 bytes 2021-02-20T08:19:15.094 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 820896768 bytes 2021-02-20T08:19:15.371 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 871257088 bytes 2021-02-20T08:19:15.605 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 897181696 bytes 2021-02-20T08:19:15.858 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 934727680 bytes 2021-02-20T08:19:16.162 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 991497216 bytes 2021-02-20T08:19:16.402 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 1019694080 bytes 2021-02-20T08:19:16.632 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 1044892672 bytes 2021-02-20T08:19:16.948 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2700 IOs, 1122792448 bytes 2021-02-20T08:19:17.257 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2800 IOs, 1196033024 bytes 2021-02-20T08:19:17.554 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2900 IOs, 1233693696 bytes 2021-02-20T08:19:17.793 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3000 IOs, 1257123840 bytes 2021-02-20T08:19:18.017 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3100 IOs, 1287539712 bytes 2021-02-20T08:19:18.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:19:18.195 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T08:19:18.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T08:19:18.196 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T08:19:18.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T08:19:18.198 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T08:19:18.269 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3200 IOs, 1321776128 bytes 2021-02-20T08:19:18.608 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3300 IOs, 1402542080 bytes 2021-02-20T08:19:19.062 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3400 IOs, 1458262016 bytes 2021-02-20T08:19:19.372 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3500 IOs, 1516900352 bytes 2021-02-20T08:19:19.723 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3600 IOs, 1600160768 bytes 2021-02-20T08:19:19.966 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3700 IOs, 1628240896 bytes 2021-02-20T08:19:20.195 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3800 IOs, 1667807232 bytes 2021-02-20T08:19:20.247 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:19:20.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:19:20.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:19:20.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:19:20.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_12 2021-02-20T08:19:20.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_12 snapshot 128M 2021-02-20T08:19:20.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:19:20.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:20.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:19:20.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:20.250 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_12 2021-02-20T08:19:20.250 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:20.250 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:19:20.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:19:20.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:20.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_12 128M 2021-02-20T08:19:20.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:19:20.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:20.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:19:20.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:20.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_12 2021-02-20T08:19:20.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:20.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:19:20.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:19:20.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:19:20.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:20.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_12 2021-02-20T08:19:20.263 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T08:19:20.264 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T08:19:20.264 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T08:19:20.265 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T08:19:20.258+0000 7faa5aa38700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 24248) UID: 1000 2021-02-20T08:19:20.465 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:19:20.566 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:19:20.567 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_12 snapshot 2021-02-20T08:19:20.567 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:19:20.567 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:19:20.567 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_12 2021-02-20T08:19:20.567 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:19:20.568 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_12 snapshot 2021-02-20T08:19:21.169 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:19:21.200 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:19:21.200 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_12 2021-02-20T08:19:21.200 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:19:21.201 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:19:21.201 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_12 2021-02-20T08:19:21.201 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:19:21.201 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=5 2021-02-20T08:19:21.202 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:19:21.639 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_12 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:19:22.137 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 23842816 bytes 2021-02-20T08:19:22.314 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 27205632 bytes 2021-02-20T08:19:22.547 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 56296448 bytes 2021-02-20T08:19:22.790 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 91923456 bytes 2021-02-20T08:19:22.997 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 106610688 bytes 2021-02-20T08:19:23.168 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 107214848 bytes 2021-02-20T08:19:23.536 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 202683392 bytes 2021-02-20T08:19:23.842 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 250178560 bytes 2021-02-20T08:19:24.365 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 280686592 bytes 2021-02-20T08:19:24.666 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 348088320 bytes 2021-02-20T08:19:24.911 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 379049984 bytes 2021-02-20T08:19:25.193 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 426887168 bytes 2021-02-20T08:19:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 464809984 bytes 2021-02-20T08:19:25.677 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 484547584 bytes 2021-02-20T08:19:25.914 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 510769152 bytes 2021-02-20T08:19:26.202 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 573389824 bytes 2021-02-20T08:19:26.493 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 621494272 bytes 2021-02-20T08:19:26.644 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:19:26.645 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:19:26.645 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:19:26.645 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:19:26.645 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_13 2021-02-20T08:19:26.646 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_13 snapshot 128M 2021-02-20T08:19:26.646 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:19:26.646 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:26.646 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:19:26.646 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:26.647 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_13 2021-02-20T08:19:26.647 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:26.647 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:19:26.647 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:19:26.647 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:26.648 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_13 128M 2021-02-20T08:19:26.648 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:19:26.648 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:26.648 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:19:26.649 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:26.649 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_13 2021-02-20T08:19:26.649 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:26.649 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:19:26.650 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:19:26.650 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:19:26.650 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:26.650 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_13 2021-02-20T08:19:26.723 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:19:26.724 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_13 snapshot 2021-02-20T08:19:26.724 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:19:26.724 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:19:26.725 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_13 2021-02-20T08:19:26.725 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:19:26.725 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_13 snapshot 2021-02-20T08:19:27.229 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:19:27.257 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:19:27.258 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_13 2021-02-20T08:19:27.258 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:19:27.258 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:19:27.259 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_13 2021-02-20T08:19:27.259 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:19:27.259 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=8 2021-02-20T08:19:27.260 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:19:27.260 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_13 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:19:27.751 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 34146304 bytes 2021-02-20T08:19:27.941 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 34767872 bytes 2021-02-20T08:19:28.217 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 81041408 bytes 2021-02-20T08:19:28.265 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T08:19:28.266 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:19:28.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:19:28.267 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T08:19:28.267 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T08:19:28.267 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T08:19:28.278 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T08:19:28.280 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T08:19:28.414 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 81651712 bytes 2021-02-20T08:19:28.673 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 117242880 bytes 2021-02-20T08:19:28.936 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 150668288 bytes 2021-02-20T08:19:29.274 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 216672256 bytes 2021-02-20T08:19:29.461 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 217299968 bytes 2021-02-20T08:19:29.755 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 286418944 bytes 2021-02-20T08:19:30.078 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 351019008 bytes 2021-02-20T08:19:30.414 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 451950592 bytes 2021-02-20T08:19:30.660 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 488098816 bytes 2021-02-20T08:19:30.960 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 524156928 bytes 2021-02-20T08:19:31.197 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 551309312 bytes 2021-02-20T08:19:31.468 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 596587520 bytes 2021-02-20T08:19:31.642 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 597193728 bytes 2021-02-20T08:19:31.937 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 676759552 bytes 2021-02-20T08:19:32.321 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 744024064 bytes 2021-02-20T08:19:32.510 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 752358400 bytes 2021-02-20T08:19:32.772 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 794838016 bytes 2021-02-20T08:19:33.082 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 864360448 bytes 2021-02-20T08:19:33.384 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 917091328 bytes 2021-02-20T08:19:33.665 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 958497792 bytes 2021-02-20T08:19:33.993 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 1023561728 bytes 2021-02-20T08:19:34.175 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 1024162816 bytes 2021-02-20T08:19:34.547 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 1084982272 bytes 2021-02-20T08:19:35.032 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2700 IOs, 1193381888 bytes 2021-02-20T08:19:35.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2800 IOs, 1203009536 bytes 2021-02-20T08:19:35.265 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:19:35.266 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:19:35.266 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:19:35.267 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:19:35.267 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_14 2021-02-20T08:19:35.267 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_14 snapshot 128M 2021-02-20T08:19:35.268 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:19:35.268 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:35.269 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:19:35.269 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:35.270 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_14 2021-02-20T08:19:35.270 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:35.271 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:19:35.271 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:19:35.271 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:35.272 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_14 128M 2021-02-20T08:19:35.272 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:19:35.273 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:35.273 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:19:35.274 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:35.274 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_14 2021-02-20T08:19:35.275 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:35.275 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:19:35.275 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:19:35.276 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:19:35.276 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:35.277 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_14 2021-02-20T08:19:35.341 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:19:35.342 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_14 snapshot 2021-02-20T08:19:35.342 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:19:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:19:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_14 2021-02-20T08:19:35.344 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:19:35.345 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_14 snapshot 2021-02-20T08:19:36.191 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:19:36.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:19:36.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_14 2021-02-20T08:19:36.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:19:36.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:19:36.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_14 2021-02-20T08:19:36.226 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:19:36.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=12 2021-02-20T08:19:36.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:19:36.227 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_14 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:19:36.727 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 30790656 bytes 2021-02-20T08:19:36.994 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 86078464 bytes 2021-02-20T08:19:37.204 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 97447936 bytes 2021-02-20T08:19:37.471 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 142234624 bytes 2021-02-20T08:19:37.782 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 210614272 bytes 2021-02-20T08:19:38.005 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 229710848 bytes 2021-02-20T08:19:38.285 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 269082624 bytes 2021-02-20T08:19:38.484 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 275762176 bytes 2021-02-20T08:19:38.663 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 276367360 bytes 2021-02-20T08:19:38.883 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 289210368 bytes 2021-02-20T08:19:39.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 332053504 bytes 2021-02-20T08:19:39.658 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 348518400 bytes 2021-02-20T08:19:40.002 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 418395136 bytes 2021-02-20T08:19:40.194 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 425185280 bytes 2021-02-20T08:19:40.516 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 492344320 bytes 2021-02-20T08:19:40.887 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 596142080 bytes 2021-02-20T08:19:41.126 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 621234176 bytes 2021-02-20T08:19:41.402 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 672282624 bytes 2021-02-20T08:19:41.667 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 712430592 bytes 2021-02-20T08:19:42.179 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 810275840 bytes 2021-02-20T08:19:42.435 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 845916160 bytes 2021-02-20T08:19:42.692 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 885022720 bytes 2021-02-20T08:19:42.912 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 912407552 bytes 2021-02-20T08:19:43.154 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 950679552 bytes 2021-02-20T08:19:43.386 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 962442240 bytes 2021-02-20T08:19:43.622 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 989684736 bytes 2021-02-20T08:19:43.842 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2700 IOs, 1015775232 bytes 2021-02-20T08:19:44.704 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2800 IOs, 1122412544 bytes 2021-02-20T08:19:44.988 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2900 IOs, 1165996032 bytes 2021-02-20T08:19:45.313 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3000 IOs, 1240597504 bytes 2021-02-20T08:19:45.599 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3100 IOs, 1293693952 bytes 2021-02-20T08:19:45.962 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3200 IOs, 1381070848 bytes 2021-02-20T08:19:46.209 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3300 IOs, 1412146176 bytes 2021-02-20T08:19:46.540 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3400 IOs, 1478276096 bytes 2021-02-20T08:19:46.731 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3500 IOs, 1479174144 bytes 2021-02-20T08:19:46.942 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3600 IOs, 1494646784 bytes 2021-02-20T08:19:47.162 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3700 IOs, 1519163392 bytes 2021-02-20T08:19:47.555 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3800 IOs, 1621230592 bytes 2021-02-20T08:19:47.806 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3900 IOs, 1656748032 bytes 2021-02-20T08:19:48.016 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4000 IOs, 1666361344 bytes 2021-02-20T08:19:48.238 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:19:48.238 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:19:48.238 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:19:48.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:19:48.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_15 2021-02-20T08:19:48.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_15 snapshot 128M 2021-02-20T08:19:48.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:19:48.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:48.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:19:48.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:48.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_15 2021-02-20T08:19:48.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:48.241 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:19:48.241 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:19:48.241 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:48.241 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_15 128M 2021-02-20T08:19:48.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:19:48.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:48.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:19:48.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:48.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_15 2021-02-20T08:19:48.243 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:48.243 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:19:48.243 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:19:48.243 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:19:48.243 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:19:48.244 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_15 2021-02-20T08:19:48.308 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:19:48.308 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_15 snapshot 2021-02-20T08:19:48.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:19:48.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:19:48.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_15 2021-02-20T08:19:48.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:19:48.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_15 snapshot 2021-02-20T08:19:49.728 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:19:49.757 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:19:49.757 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_15 2021-02-20T08:19:49.758 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:19:49.758 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:19:49.758 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_15 2021-02-20T08:19:49.758 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:19:49.759 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=12 2021-02-20T08:19:49.759 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:19:49.759 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_15 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:19:50.432 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 66314240 bytes 2021-02-20T08:19:50.665 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 97420288 bytes 2021-02-20T08:19:50.971 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 180470784 bytes 2021-02-20T08:19:51.222 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 208343040 bytes 2021-02-20T08:19:51.519 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 266355712 bytes 2021-02-20T08:19:51.702 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 266958848 bytes 2021-02-20T08:19:52.005 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 334555136 bytes 2021-02-20T08:19:52.266 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 356446208 bytes 2021-02-20T08:19:52.508 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 384954368 bytes 2021-02-20T08:19:52.777 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 431862784 bytes 2021-02-20T08:19:53.077 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 487927808 bytes 2021-02-20T08:19:53.398 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 554741760 bytes 2021-02-20T08:19:53.718 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 627446784 bytes 2021-02-20T08:19:53.989 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 672671744 bytes 2021-02-20T08:19:54.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 740029440 bytes 2021-02-20T08:19:54.541 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 761131008 bytes 2021-02-20T08:19:54.970 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 796876800 bytes 2021-02-20T08:19:55.245 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 847083520 bytes 2021-02-20T08:19:55.492 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 894901248 bytes 2021-02-20T08:19:55.751 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 942446592 bytes 2021-02-20T08:19:56.190 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 953651200 bytes 2021-02-20T08:19:56.417 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 989428736 bytes 2021-02-20T08:19:56.665 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 1015507968 bytes 2021-02-20T08:19:56.905 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 1040616448 bytes 2021-02-20T08:19:57.225 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 1114319872 bytes 2021-02-20T08:19:57.472 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 1140773888 bytes 2021-02-20T08:19:57.748 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2700 IOs, 1186300928 bytes 2021-02-20T08:19:58.019 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2800 IOs, 1228038144 bytes 2021-02-20T08:19:58.348 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2900 IOs, 1292915712 bytes 2021-02-20T08:19:58.679 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3000 IOs, 1355802624 bytes 2021-02-20T08:19:58.909 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3100 IOs, 1384402944 bytes 2021-02-20T08:19:59.265 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3200 IOs, 1459657728 bytes 2021-02-20T08:19:59.583 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3300 IOs, 1525850112 bytes 2021-02-20T08:19:59.941 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3400 IOs, 1542284288 bytes 2021-02-20T08:20:00.208 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3500 IOs, 1588946944 bytes 2021-02-20T08:20:00.411 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3600 IOs, 1600525312 bytes 2021-02-20T08:20:00.765 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3700 IOs, 1612429312 bytes 2021-02-20T08:20:01.089 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3800 IOs, 1684997120 bytes 2021-02-20T08:20:01.409 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3900 IOs, 1755577344 bytes 2021-02-20T08:20:01.596 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:4000 IOs, 1756201984 bytes 2021-02-20T08:20:01.763 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:20:01.764 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:20:01.765 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:20:01.765 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:20:01.766 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_16 2021-02-20T08:20:01.766 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_16 snapshot 128M 2021-02-20T08:20:01.766 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:01.766 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:01.766 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:01.767 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:01.767 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_16 2021-02-20T08:20:01.767 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:01.767 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:20:01.768 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:20:01.768 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:01.768 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_16 128M 2021-02-20T08:20:01.768 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:01.768 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:01.769 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:01.769 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:01.769 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_16 2021-02-20T08:20:01.769 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:01.770 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:20:01.770 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:20:01.770 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:20:01.770 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:01.770 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_16 2021-02-20T08:20:01.837 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:20:01.837 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_16 snapshot 2021-02-20T08:20:01.838 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:01.838 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:01.838 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_16 2021-02-20T08:20:01.839 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:20:01.839 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_16 snapshot 2021-02-20T08:20:02.206 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:20:02.238 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:20:02.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_16 2021-02-20T08:20:02.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:02.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:02.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_16 2021-02-20T08:20:02.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:20:02.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=12 2021-02-20T08:20:02.241 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:20:02.241 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_16 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:20:02.676 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 12101632 bytes 2021-02-20T08:20:02.937 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 50887680 bytes 2021-02-20T08:20:03.208 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 89968640 bytes 2021-02-20T08:20:03.495 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 146088960 bytes 2021-02-20T08:20:03.752 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 189960192 bytes 2021-02-20T08:20:04.005 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 210442240 bytes 2021-02-20T08:20:04.599 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 291895296 bytes 2021-02-20T08:20:05.132 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 297127936 bytes 2021-02-20T08:20:05.795 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 351277056 bytes 2021-02-20T08:20:06.332 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 384977920 bytes 2021-02-20T08:20:06.959 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 474364928 bytes 2021-02-20T08:20:07.430 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 504183808 bytes 2021-02-20T08:20:07.938 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 527405056 bytes 2021-02-20T08:20:08.466 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 560956416 bytes 2021-02-20T08:20:08.888 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 592669696 bytes 2021-02-20T08:20:09.351 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 645708800 bytes 2021-02-20T08:20:09.804 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 646326272 bytes 2021-02-20T08:20:10.305 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 715661312 bytes 2021-02-20T08:20:10.761 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 730359808 bytes 2021-02-20T08:20:11.238 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 746584064 bytes 2021-02-20T08:20:11.742 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 810600448 bytes 2021-02-20T08:20:12.281 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 864982016 bytes 2021-02-20T08:20:12.793 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 984033280 bytes 2021-02-20T08:20:13.187 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 1007515648 bytes 2021-02-20T08:20:13.579 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 1050865664 bytes 2021-02-20T08:20:14.058 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 1104850944 bytes 2021-02-20T08:20:14.244 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:20:14.244 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:20:14.244 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:20:14.245 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:20:14.245 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_17 2021-02-20T08:20:14.245 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_17 snapshot 128M 2021-02-20T08:20:14.245 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:14.246 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:14.246 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:14.246 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:14.246 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_17 2021-02-20T08:20:14.246 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:14.247 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:20:14.247 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:20:14.247 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:14.247 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_17 128M 2021-02-20T08:20:14.247 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:14.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:14.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:14.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:14.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_17 2021-02-20T08:20:14.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:14.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:20:14.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:20:14.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:20:14.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:14.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_17 2021-02-20T08:20:14.458 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:20:14.458 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_17 snapshot 2021-02-20T08:20:14.458 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:14.459 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:14.459 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_17 2021-02-20T08:20:14.459 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:20:14.459 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_17 snapshot 2021-02-20T08:20:15.221 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:20:15.260 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:20:15.260 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_17 2021-02-20T08:20:15.260 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:15.260 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:15.261 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_17 2021-02-20T08:20:15.261 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:20:15.262 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=7 2021-02-20T08:20:15.262 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:20:15.262 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_17 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:20:15.830 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 60131328 bytes 2021-02-20T08:20:16.129 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 133441536 bytes 2021-02-20T08:20:16.387 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 172982272 bytes 2021-02-20T08:20:16.606 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 173593600 bytes 2021-02-20T08:20:17.085 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 223295488 bytes 2021-02-20T08:20:17.158 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 257204224 bytes 2021-02-20T08:20:17.522 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 312173568 bytes 2021-02-20T08:20:17.926 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 376678400 bytes 2021-02-20T08:20:18.299 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 417268736 bytes 2021-02-20T08:20:18.654 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 456620032 bytes 2021-02-20T08:20:19.044 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 485719040 bytes 2021-02-20T08:20:19.434 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 522805248 bytes 2021-02-20T08:20:19.850 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 536019968 bytes 2021-02-20T08:20:20.305 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 576832512 bytes 2021-02-20T08:20:20.692 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 636963840 bytes 2021-02-20T08:20:21.125 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 679546880 bytes 2021-02-20T08:20:21.548 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 702610432 bytes 2021-02-20T08:20:21.972 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 743967744 bytes 2021-02-20T08:20:22.474 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:20:22.474 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:20:22.475 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:20:22.475 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:20:22.475 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_18 2021-02-20T08:20:22.475 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_18 snapshot 128M 2021-02-20T08:20:22.476 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:22.476 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:22.476 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:22.476 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:22.477 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_18 2021-02-20T08:20:22.477 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:22.477 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:20:22.477 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:20:22.478 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:22.478 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_18 128M 2021-02-20T08:20:22.478 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:22.479 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:22.479 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:22.479 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:22.479 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_18 2021-02-20T08:20:22.479 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:22.480 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:20:22.480 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:20:22.480 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:20:22.480 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:22.481 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_18 2021-02-20T08:20:22.716 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:20:22.717 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_18 snapshot 2021-02-20T08:20:22.717 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:22.717 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:22.717 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_18 2021-02-20T08:20:22.717 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:20:22.718 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_18 snapshot 2021-02-20T08:20:23.340 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:20:23.377 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:20:23.378 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_18 2021-02-20T08:20:23.378 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:23.378 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:23.378 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_18 2021-02-20T08:20:23.379 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:20:23.379 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=7 2021-02-20T08:20:23.380 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:20:23.380 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_18 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:20:23.998 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 17617920 bytes 2021-02-20T08:20:24.129 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 63120384 bytes 2021-02-20T08:20:24.500 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 117365760 bytes 2021-02-20T08:20:24.983 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 142120960 bytes 2021-02-20T08:20:25.614 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 174302208 bytes 2021-02-20T08:20:25.964 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 227935232 bytes 2021-02-20T08:20:26.311 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 285380608 bytes 2021-02-20T08:20:26.697 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 357408768 bytes 2021-02-20T08:20:27.198 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 443039744 bytes 2021-02-20T08:20:27.716 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 520544256 bytes 2021-02-20T08:20:28.159 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 537919488 bytes 2021-02-20T08:20:28.458 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 590339072 bytes 2021-02-20T08:20:28.872 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 616676352 bytes 2021-02-20T08:20:29.286 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 653576192 bytes 2021-02-20T08:20:29.909 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 745126912 bytes 2021-02-20T08:20:30.231 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 756510720 bytes 2021-02-20T08:20:30.409 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:20:30.410 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:20:30.410 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:20:30.410 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:20:30.410 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_19 2021-02-20T08:20:30.411 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_19 snapshot 128M 2021-02-20T08:20:30.411 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:30.411 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:30.411 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:30.412 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:30.412 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_19 2021-02-20T08:20:30.412 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:30.412 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:20:30.413 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:20:30.413 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:30.413 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_19 128M 2021-02-20T08:20:30.413 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:30.414 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:30.414 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:30.414 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:30.414 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_19 2021-02-20T08:20:30.415 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:30.415 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:20:30.415 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:20:30.415 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:20:30.416 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:30.416 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_19 2021-02-20T08:20:30.502 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:20:30.502 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_19 snapshot 2021-02-20T08:20:30.502 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:30.503 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:30.503 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_19 2021-02-20T08:20:30.503 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:20:30.503 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_19 snapshot 2021-02-20T08:20:31.208 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:20:31.243 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:20:31.243 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_19 2021-02-20T08:20:31.244 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:31.244 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:31.244 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_19 2021-02-20T08:20:31.244 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:20:31.245 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=11 2021-02-20T08:20:31.245 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:20:31.246 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_19 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:20:31.783 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 63597568 bytes 2021-02-20T08:20:32.016 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 95347712 bytes 2021-02-20T08:20:32.313 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 145438720 bytes 2021-02-20T08:20:32.549 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 169852928 bytes 2021-02-20T08:20:32.816 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 216230912 bytes 2021-02-20T08:20:33.189 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 269732864 bytes 2021-02-20T08:20:33.545 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 308641792 bytes 2021-02-20T08:20:34.059 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 409734144 bytes 2021-02-20T08:20:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 428961792 bytes 2021-02-20T08:20:34.880 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 442370048 bytes 2021-02-20T08:20:35.287 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 442979328 bytes 2021-02-20T08:20:35.943 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 556596224 bytes 2021-02-20T08:20:36.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 593716224 bytes 2021-02-20T08:20:36.601 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 605292544 bytes 2021-02-20T08:20:37.168 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 670338048 bytes 2021-02-20T08:20:37.510 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 713680896 bytes 2021-02-20T08:20:37.942 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 720403456 bytes 2021-02-20T08:20:38.403 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 769190912 bytes 2021-02-20T08:20:38.880 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 842561536 bytes 2021-02-20T08:20:39.318 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 887879680 bytes 2021-02-20T08:20:39.756 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 889320448 bytes 2021-02-20T08:20:40.169 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 933584896 bytes 2021-02-20T08:20:40.736 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 1074292736 bytes 2021-02-20T08:20:41.209 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 1122212864 bytes 2021-02-20T08:20:41.675 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 1174282240 bytes 2021-02-20T08:20:42.232 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 1273093120 bytes 2021-02-20T08:20:42.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:20:42.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:20:42.250 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:20:42.250 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:20:42.250 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_20 2021-02-20T08:20:42.250 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_20 snapshot 128M 2021-02-20T08:20:42.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:42.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:42.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:42.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:42.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_20 2021-02-20T08:20:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:20:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:20:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_20 128M 2021-02-20T08:20:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:42.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_20 2021-02-20T08:20:42.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:42.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:20:42.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:20:42.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:20:42.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:42.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_20 2021-02-20T08:20:42.407 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:20:42.407 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_20 snapshot 2021-02-20T08:20:42.407 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:42.407 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:42.408 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_20 2021-02-20T08:20:42.408 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:20:42.408 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_20 snapshot 2021-02-20T08:20:43.254 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:20:43.285 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:20:43.285 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_20 2021-02-20T08:20:43.286 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:43.286 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:43.286 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_20 2021-02-20T08:20:43.286 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:20:43.287 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=8 2021-02-20T08:20:43.287 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:20:43.288 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_20 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:20:43.836 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 45273088 bytes 2021-02-20T08:20:44.242 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 147041280 bytes 2021-02-20T08:20:44.575 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 222295040 bytes 2021-02-20T08:20:44.839 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 251279360 bytes 2021-02-20T08:20:45.144 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 295792640 bytes 2021-02-20T08:20:45.457 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 317872128 bytes 2021-02-20T08:20:45.865 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 368162816 bytes 2021-02-20T08:20:46.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 421283840 bytes 2021-02-20T08:20:46.364 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 443973632 bytes 2021-02-20T08:20:46.757 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 529718272 bytes 2021-02-20T08:20:46.963 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 538070016 bytes 2021-02-20T08:20:47.342 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 546975744 bytes 2021-02-20T08:20:47.745 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 550111232 bytes 2021-02-20T08:20:48.173 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 595609600 bytes 2021-02-20T08:20:48.539 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 602957824 bytes 2021-02-20T08:20:49.064 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 664795136 bytes 2021-02-20T08:20:49.513 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 718559232 bytes 2021-02-20T08:20:49.992 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 781414400 bytes 2021-02-20T08:20:50.513 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 805830656 bytes 2021-02-20T08:20:50.849 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 842982400 bytes 2021-02-20T08:20:51.255 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 874534912 bytes 2021-02-20T08:20:51.291 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:20:51.292 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:20:51.292 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:20:51.292 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:20:51.292 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_21 2021-02-20T08:20:51.292 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_21 snapshot 128M 2021-02-20T08:20:51.293 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:51.293 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:51.293 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:51.293 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:51.293 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_21 2021-02-20T08:20:51.294 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:51.294 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:20:51.294 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:20:51.294 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:51.295 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_21 128M 2021-02-20T08:20:51.295 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:51.295 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:51.295 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:51.295 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:51.296 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_21 2021-02-20T08:20:51.296 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:51.296 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:20:51.296 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:20:51.296 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:20:51.297 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:20:51.297 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_21 2021-02-20T08:20:51.467 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:20:51.467 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_21 snapshot 2021-02-20T08:20:51.468 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:51.468 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:51.468 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_21 2021-02-20T08:20:51.468 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:20:51.469 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_21 snapshot 2021-02-20T08:20:52.209 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:20:52.238 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:20:52.238 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_21 2021-02-20T08:20:52.238 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:20:52.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:20:52.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_21 2021-02-20T08:20:52.239 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:20:52.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=8 2021-02-20T08:20:52.240 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:20:52.241 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_21 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:20:52.718 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 24986624 bytes 2021-02-20T08:20:52.970 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 62270464 bytes 2021-02-20T08:20:53.191 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 91094016 bytes 2021-02-20T08:20:53.434 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 120578048 bytes 2021-02-20T08:20:53.764 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 193018880 bytes 2021-02-20T08:20:54.035 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 236942336 bytes 2021-02-20T08:20:54.356 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 306266112 bytes 2021-02-20T08:20:54.650 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 352696320 bytes 2021-02-20T08:20:54.944 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 391894016 bytes 2021-02-20T08:20:55.386 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 500105216 bytes 2021-02-20T08:20:55.778 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 523448320 bytes 2021-02-20T08:20:56.275 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 598330368 bytes 2021-02-20T08:20:56.687 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 626422784 bytes 2021-02-20T08:20:57.128 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 669054976 bytes 2021-02-20T08:20:57.548 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 709561344 bytes 2021-02-20T08:20:58.007 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 754434048 bytes 2021-02-20T08:20:58.419 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 755031040 bytes 2021-02-20T08:20:58.950 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 864111616 bytes 2021-02-20T08:20:59.361 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 913378304 bytes 2021-02-20T08:20:59.726 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 933844992 bytes 2021-02-20T08:21:00.191 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 985904128 bytes 2021-02-20T08:21:00.246 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:21:00.247 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:21:00.247 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:21:00.247 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:21:00.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_22 2021-02-20T08:21:00.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_22 snapshot 128M 2021-02-20T08:21:00.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:21:00.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:00.248 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:21:00.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:00.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_22 2021-02-20T08:21:00.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:00.249 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:21:00.250 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:21:00.250 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:00.250 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_22 128M 2021-02-20T08:21:00.250 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:21:00.250 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:00.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:21:00.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:00.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_22 2021-02-20T08:21:00.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:00.251 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:21:00.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:21:00.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:21:00.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:00.252 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_22 2021-02-20T08:21:01.146 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:21:01.146 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_22 snapshot 2021-02-20T08:21:01.146 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:21:01.147 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:21:01.147 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_22 2021-02-20T08:21:01.147 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:21:01.147 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_22 snapshot 2021-02-20T08:21:02.232 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:21:02.263 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:21:02.264 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_22 2021-02-20T08:21:02.264 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:21:02.265 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:21:02.265 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_22 2021-02-20T08:21:02.265 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:21:02.266 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=11 2021-02-20T08:21:02.266 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:21:02.266 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_22 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:21:02.827 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 44651520 bytes 2021-02-20T08:21:03.111 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 94946304 bytes 2021-02-20T08:21:03.311 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 105340928 bytes 2021-02-20T08:21:03.628 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 164530176 bytes 2021-02-20T08:21:03.924 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 198766592 bytes 2021-02-20T08:21:04.276 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 246991872 bytes 2021-02-20T08:21:04.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9141487975575567) > weight (0.25) 2021-02-20T08:21:04.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.42801721440947094) > weight (0.25) 2021-02-20T08:21:04.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9534695904580032) > weight (0.25) 2021-02-20T08:21:04.283 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3892136782085779) > weight (0.25) 2021-02-20T08:21:04.283 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T08:21:04.576 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 267385856 bytes 2021-02-20T08:21:04.894 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 295631872 bytes 2021-02-20T08:21:05.275 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 372465664 bytes 2021-02-20T08:21:05.568 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 410424320 bytes 2021-02-20T08:21:05.948 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 490268672 bytes 2021-02-20T08:21:06.461 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 522819584 bytes 2021-02-20T08:21:06.779 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 581694464 bytes 2021-02-20T08:21:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 656359424 bytes 2021-02-20T08:21:07.199 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T08:21:07.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T08:21:07.201 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T08:21:07.343 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 656972800 bytes 2021-02-20T08:21:07.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T08:21:07.384+0000 7f0dff34f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 26411) UID: 1000 2021-02-20T08:21:07.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:21:07.901 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 695623680 bytes 2021-02-20T08:21:08.175 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 734508032 bytes 2021-02-20T08:21:08.408 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 755887104 bytes 2021-02-20T08:21:08.668 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 796170240 bytes 2021-02-20T08:21:09.003 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 847352832 bytes 2021-02-20T08:21:09.374 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 907032576 bytes 2021-02-20T08:21:09.640 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 914598912 bytes 2021-02-20T08:21:09.922 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 925162496 bytes 2021-02-20T08:21:10.293 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 994363392 bytes 2021-02-20T08:21:10.626 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 1012089856 bytes 2021-02-20T08:21:11.012 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 1054150656 bytes 2021-02-20T08:21:11.395 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2700 IOs, 1054771200 bytes 2021-02-20T08:21:11.844 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2800 IOs, 1099701248 bytes 2021-02-20T08:21:12.274 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2900 IOs, 1169754112 bytes 2021-02-20T08:21:12.732 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3000 IOs, 1234945024 bytes 2021-02-20T08:21:13.103 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3100 IOs, 1235565568 bytes 2021-02-20T08:21:13.202 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T08:21:13.202 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:21:13.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:21:13.203 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T08:21:13.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T08:21:13.204 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T08:21:13.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T08:21:13.439 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T08:21:13.439 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:21:13.440 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:21:13.440 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:21:13.440 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:21:13.440 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_23 2021-02-20T08:21:13.441 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_23 snapshot 128M 2021-02-20T08:21:13.441 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:21:13.441 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:13.441 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:21:13.441 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:13.442 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_23 2021-02-20T08:21:13.442 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:13.442 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:21:13.442 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:21:13.443 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:13.443 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_23 128M 2021-02-20T08:21:13.443 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:21:13.443 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:13.443 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:21:13.444 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:13.444 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_23 2021-02-20T08:21:13.444 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:13.444 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:21:13.444 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:21:13.445 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:21:13.446 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:13.446 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_23 2021-02-20T08:21:13.749 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:21:13.749 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_23 snapshot 2021-02-20T08:21:13.749 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:21:13.749 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:21:13.750 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_23 2021-02-20T08:21:13.750 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:21:13.750 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_23 snapshot 2021-02-20T08:21:14.263 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:21:14.299 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:21:14.299 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_23 2021-02-20T08:21:14.299 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:21:14.300 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:21:14.300 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_23 2021-02-20T08:21:14.300 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:21:14.301 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=7 2021-02-20T08:21:14.301 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:21:14.301 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_23 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:21:14.837 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 42591232 bytes 2021-02-20T08:21:15.090 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 72392704 bytes 2021-02-20T08:21:15.364 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 113800192 bytes 2021-02-20T08:21:15.622 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 142941184 bytes 2021-02-20T08:21:15.908 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 176036864 bytes 2021-02-20T08:21:16.259 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 178061312 bytes 2021-02-20T08:21:16.730 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 284636160 bytes 2021-02-20T08:21:17.143 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 327408640 bytes 2021-02-20T08:21:17.505 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 334947328 bytes 2021-02-20T08:21:17.875 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 337577984 bytes 2021-02-20T08:21:18.315 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 378435584 bytes 2021-02-20T08:21:18.780 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 428850176 bytes 2021-02-20T08:21:19.401 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 581595136 bytes 2021-02-20T08:21:19.753 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 586048512 bytes 2021-02-20T08:21:20.152 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 612219904 bytes 2021-02-20T08:21:20.547 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 628946944 bytes 2021-02-20T08:21:21.029 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 710616064 bytes 2021-02-20T08:21:21.305 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ error_code=124 2021-02-20T08:21:21.306 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set -e 2021-02-20T08:21:21.306 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:21:21.306 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ return 0 2021-02-20T08:21:21.307 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ image=image_24 2021-02-20T08:21:21.307 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image_and_enable_mirror cluster2 mirror image_24 snapshot 128M 2021-02-20T08:21:21.307 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:21:21.307 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:21.307 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:21:21.308 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:21.308 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_24 2021-02-20T08:21:21.308 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:21.308 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:21:21.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n snapshot ] 2021-02-20T08:21:21.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:21.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ create_image cluster2 mirror image_24 128M 2021-02-20T08:21:21.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:21:21.309 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:21.310 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:21:21.310 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:21.310 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_24 2021-02-20T08:21:21.310 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:21.311 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local size=128 2021-02-20T08:21:21.311 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n 128M ] 2021-02-20T08:21:21.311 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ size=128M 2021-02-20T08:21:21.311 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ shift 2021-02-20T08:21:21.311 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_24 2021-02-20T08:21:21.444 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ image = image ] 2021-02-20T08:21:21.445 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ enable_mirror cluster2 mirror image_24 snapshot 2021-02-20T08:21:21.445 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:21:21.445 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:21:21.445 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_24 2021-02-20T08:21:21.446 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local mode=snapshot 2021-02-20T08:21:21.446 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_24 snapshot 2021-02-20T08:21:22.237 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:Mirroring enabled 2021-02-20T08:21:22.266 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ [ -n ] 2021-02-20T08:21:22.267 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ stress_write_image cluster2 mirror image_24 2021-02-20T08:21:22.267 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local cluster=cluster2 2021-02-20T08:21:22.267 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local pool=mirror 2021-02-20T08:21:22.267 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local image=image_24 2021-02-20T08:21:22.268 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:21:22.268 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ local duration=12 2021-02-20T08:21:22.269 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ set +e 2021-02-20T08:21:22.269 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_24 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:21:22.872 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:100 IOs, 86893568 bytes 2021-02-20T08:21:23.291 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:200 IOs, 198619136 bytes 2021-02-20T08:21:23.559 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:300 IOs, 242799616 bytes 2021-02-20T08:21:23.778 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:400 IOs, 254597120 bytes 2021-02-20T08:21:24.123 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:500 IOs, 328127488 bytes 2021-02-20T08:21:24.356 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:600 IOs, 336099328 bytes 2021-02-20T08:21:24.596 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:700 IOs, 345989120 bytes 2021-02-20T08:21:24.882 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:800 IOs, 367308800 bytes 2021-02-20T08:21:25.311 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:900 IOs, 437061632 bytes 2021-02-20T08:21:25.602 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1000 IOs, 444593152 bytes 2021-02-20T08:21:25.916 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1100 IOs, 456085504 bytes 2021-02-20T08:21:26.409 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1200 IOs, 513372160 bytes 2021-02-20T08:21:26.839 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1300 IOs, 552392704 bytes 2021-02-20T08:21:27.228 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1400 IOs, 584602624 bytes 2021-02-20T08:21:27.631 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1500 IOs, 611989504 bytes 2021-02-20T08:21:28.027 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1600 IOs, 626445312 bytes 2021-02-20T08:21:28.462 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1700 IOs, 662609920 bytes 2021-02-20T08:21:28.837 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1800 IOs, 663233536 bytes 2021-02-20T08:21:29.301 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:1900 IOs, 717989888 bytes 2021-02-20T08:21:29.727 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2000 IOs, 750687232 bytes 2021-02-20T08:21:30.131 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2100 IOs, 797490176 bytes 2021-02-20T08:21:30.579 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2200 IOs, 808024064 bytes 2021-02-20T08:21:31.021 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2300 IOs, 829851648 bytes 2021-02-20T08:21:31.431 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2400 IOs, 850168832 bytes 2021-02-20T08:21:31.904 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2500 IOs, 872315904 bytes 2021-02-20T08:21:32.404 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2600 IOs, 974005248 bytes 2021-02-20T08:21:32.858 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2700 IOs, 1015337984 bytes 2021-02-20T08:21:33.287 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2800 IOs, 1089144832 bytes 2021-02-20T08:21:33.711 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:2900 IOs, 1108050944 bytes 2021-02-20T08:21:34.122 INFO:tasks.workunit.cluster1.client.mirror.smithi090.stdout:3000 IOs, 1163142144 bytes 2021-02-20T08:22:20.440 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4393053754754741) > weight (0.25) 2021-02-20T08:22:20.441 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9142320760663595) > weight (0.25) 2021-02-20T08:22:20.441 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T08:22:20.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T08:22:20.442 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T08:22:20.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T08:22:20.439+0000 7f0f2ba8c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 23256) UID: 1000 2021-02-20T08:22:20.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:22:23.444 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T08:22:23.444 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:22:23.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:22:23.445 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T08:22:23.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T08:22:23.446 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T08:22:23.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T08:22:23.449 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T08:23:03.284 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.954022062503361) > weight (0.25) 2021-02-20T08:23:03.285 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.43029952972453744) > weight (0.25) 2021-02-20T08:23:03.285 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.260277376024742) > weight (0.25) 2021-02-20T08:23:03.286 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9315760610627783) > weight (0.25) 2021-02-20T08:23:03.286 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T08:24:09.450 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.36104201286494064) > weight (0.25) 2021-02-20T08:24:09.451 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9956926620796686) > weight (0.25) 2021-02-20T08:24:09.451 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T08:24:09.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T08:24:09.453 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T08:24:09.453 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T08:24:09.454 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:24:09.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T08:24:09.449+0000 7fd052e57700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 29888) UID: 1000 2021-02-20T08:24:09.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:24:15.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:24:15.556 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T08:24:15.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T08:24:15.557 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T08:24:15.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T08:24:15.560 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T08:24:38.287 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5046446166790809) > weight (0.25) 2021-02-20T08:24:38.288 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2925863801512468) > weight (0.25) 2021-02-20T08:24:38.288 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.32864063207590344) > weight (0.25) 2021-02-20T08:24:38.288 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4571845911625655) > weight (0.25) 2021-02-20T08:24:38.289 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T08:25:34.562 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T08:25:34.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T08:25:34.564 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T08:25:34.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T08:25:34.560+0000 7f167a741700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 29673) UID: 1000 2021-02-20T08:25:34.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:25:35.565 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T08:25:35.566 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:25:35.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:25:35.566 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T08:25:35.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T08:25:35.567 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T08:25:35.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T08:25:35.571 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T08:26:36.289 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9927210737038085) > weight (0.25) 2021-02-20T08:26:36.290 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8976762570017139) > weight (0.25) 2021-02-20T08:26:36.291 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9080982833266087) > weight (0.25) 2021-02-20T08:26:36.291 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6676983251715302) > weight (0.25) 2021-02-20T08:26:36.291 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T08:26:43.572 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T08:26:43.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T08:26:43.574 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T08:26:43.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T08:26:43.571+0000 7f94cf1d9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 30081) UID: 1000 2021-02-20T08:26:43.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:26:48.575 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T08:26:48.575 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:26:48.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:26:48.576 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T08:26:48.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T08:26:48.577 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T08:26:48.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T08:26:48.580 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T08:27:36.292 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9509458072456963) > weight (0.25) 2021-02-20T08:27:36.293 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6148097421885278) > weight (0.25) 2021-02-20T08:27:36.293 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8361905363443767) > weight (0.25) 2021-02-20T08:27:36.293 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7728351933083205) > weight (0.25) 2021-02-20T08:27:36.294 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T08:28:01.581 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.39814059895014164) > weight (0.25) 2021-02-20T08:28:01.581 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5041200996420767) > weight (0.25) 2021-02-20T08:28:01.582 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2837237881901312) > weight (0.25) 2021-02-20T08:28:01.582 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6135213759176782) > weight (0.25) 2021-02-20T08:28:01.583 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T08:28:48.295 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7523813303532676) > weight (0.25) 2021-02-20T08:28:48.296 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7945479960185371) > weight (0.25) 2021-02-20T08:28:48.296 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T08:28:48.297 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T08:28:48.297 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T08:28:48.299 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T08:28:48.297+0000 7f684c417700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 19103) UID: 1000 2021-02-20T08:28:48.499 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:28:55.298 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T08:28:55.299 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:28:55.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:28:55.300 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T08:28:55.300 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T08:28:55.300 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T08:28:55.303 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T08:28:55.303 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T08:29:01.584 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T08:29:01.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T08:29:01.585 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T08:29:01.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T08:29:01.581+0000 7fbac3941700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 30180) UID: 1000 2021-02-20T08:29:01.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:29:09.586 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T08:29:09.586 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:29:09.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:29:09.587 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T08:29:09.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T08:29:09.588 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T08:29:09.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T08:29:09.591 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T08:30:32.305 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5855019646029809) > weight (0.25) 2021-02-20T08:30:32.306 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8677879304388946) > weight (0.25) 2021-02-20T08:30:32.306 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T08:30:32.307 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T08:30:32.307 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T08:30:32.308 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T08:30:32.308+0000 7f2d6af96700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 30273) UID: 1000 2021-02-20T08:30:32.509 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:30:37.308 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T08:30:37.309 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:30:37.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:30:37.310 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T08:30:37.311 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T08:30:37.311 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T08:30:37.315 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T08:30:37.315 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T08:30:52.592 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T08:30:52.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T08:30:52.594 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T08:30:52.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T08:30:52.592+0000 7fa67a7bb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 30364) UID: 1000 2021-02-20T08:30:52.655 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T08:30:52.650+0000 7fb533c45700 -1 reset not still connected to 0x558b7c777260 2021-02-20T08:30:52.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:30:57.595 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T08:30:57.595 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:30:57.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:30:57.596 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T08:30:57.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T08:30:57.597 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T08:30:57.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T08:30:57.600 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T08:32:26.317 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.276505574239298) > weight (0.25) 2021-02-20T08:32:26.318 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T08:32:26.319 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T08:32:26.319 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T08:32:26.320 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T08:32:26.318+0000 7fc964142700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 22297) UID: 1000 2021-02-20T08:32:26.520 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:32:28.319 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T08:32:28.320 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:32:28.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:32:28.321 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T08:32:28.321 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T08:32:28.321 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T08:32:28.324 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T08:32:28.324 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T08:32:28.608 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T08:32:28.606+0000 7f7308f93700 -1 reset not still connected to 0x55aac2359670 2021-02-20T08:32:45.600 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.932104207850207) > weight (0.25) 2021-02-20T08:32:45.601 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T08:32:45.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T08:32:45.602 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T08:32:45.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T08:32:45.602+0000 7f9fb3ef2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 28782) UID: 1000 2021-02-20T08:32:45.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:32:52.603 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T08:32:52.604 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:32:52.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:32:52.604 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T08:32:52.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T08:32:52.605 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T08:32:52.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T08:32:52.608 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T08:33:53.609 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T08:33:53.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T08:33:53.612 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T08:33:53.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T08:33:53.609+0000 7fb81c812700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 30547) UID: 1000 2021-02-20T08:33:53.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:33:55.613 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T08:33:55.613 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:33:55.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:33:55.614 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T08:33:55.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T08:33:55.615 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T08:33:55.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T08:33:55.619 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T08:34:02.326 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6724025281454307) > weight (0.25) 2021-02-20T08:34:02.326 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7004852499749518) > weight (0.25) 2021-02-20T08:34:02.326 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6395409951116475) > weight (0.25) 2021-02-20T08:34:02.327 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5746279794678391) > weight (0.25) 2021-02-20T08:34:02.327 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T08:35:14.328 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3283122432100739) > weight (0.25) 2021-02-20T08:35:14.329 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5831372925164784) > weight (0.25) 2021-02-20T08:35:14.329 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6430427739008879) > weight (0.25) 2021-02-20T08:35:14.330 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2521330883995405) > weight (0.25) 2021-02-20T08:35:14.330 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T08:35:24.620 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6436428707985528) > weight (0.25) 2021-02-20T08:35:24.621 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5622771038225842) > weight (0.25) 2021-02-20T08:35:24.621 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6362492369285669) > weight (0.25) 2021-02-20T08:35:24.621 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8049785787635336) > weight (0.25) 2021-02-20T08:35:24.621 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T08:36:30.330 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9981716704810046) > weight (0.25) 2021-02-20T08:36:30.332 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5877736532522942) > weight (0.25) 2021-02-20T08:36:30.332 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T08:36:30.333 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T08:36:30.334 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T08:36:30.335 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T08:36:30.334+0000 7f2fdca09700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 30456) UID: 1000 2021-02-20T08:36:30.535 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:36:31.334 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T08:36:31.336 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:36:31.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:36:31.337 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T08:36:31.337 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T08:36:31.338 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T08:36:31.341 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T08:36:31.341 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T08:36:44.622 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9248092600604143) > weight (0.25) 2021-02-20T08:36:44.623 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4715513622519919) > weight (0.25) 2021-02-20T08:36:44.623 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.41356327117845404) > weight (0.25) 2021-02-20T08:36:44.623 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6605164031613062) > weight (0.25) 2021-02-20T08:36:44.623 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T08:38:04.342 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.47827787018646495) > weight (0.25) 2021-02-20T08:38:04.343 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6307793747078136) > weight (0.25) 2021-02-20T08:38:04.343 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8961335955953432) > weight (0.25) 2021-02-20T08:38:04.344 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.36429926567341464) > weight (0.25) 2021-02-20T08:38:04.344 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T08:38:07.625 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T08:38:07.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T08:38:07.627 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T08:38:07.627 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T08:38:07.627 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:38:07.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T08:38:07.624+0000 7f5eb8de6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 30821) UID: 1000 2021-02-20T08:38:07.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:38:13.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:38:13.731 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T08:38:13.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T08:38:13.732 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T08:38:13.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T08:38:13.735 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T08:39:21.736 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5498692912232908) > weight (0.25) 2021-02-20T08:39:21.737 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.34315359441990645) > weight (0.25) 2021-02-20T08:39:21.737 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T08:39:21.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T08:39:21.738 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T08:39:21.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T08:39:21.739+0000 7fd3f6d10700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 29987) UID: 1000 2021-02-20T08:39:21.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:39:22.739 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T08:39:22.740 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:39:22.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:39:22.741 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T08:39:22.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T08:39:22.741 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T08:39:22.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T08:39:22.744 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T08:39:24.345 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7509659411948035) > weight (0.25) 2021-02-20T08:39:24.346 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9344952089141827) > weight (0.25) 2021-02-20T08:39:24.346 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6122100024581716) > weight (0.25) 2021-02-20T08:39:24.346 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6673036042290161) > weight (0.25) 2021-02-20T08:39:24.346 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T08:40:44.347 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.670347098545768) > weight (0.25) 2021-02-20T08:40:44.348 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T08:40:44.349 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T08:40:44.349 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T08:40:44.351 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T08:40:44.350+0000 7fa5657de700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 30638) UID: 1000 2021-02-20T08:40:44.551 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:40:51.351 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T08:40:51.359 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:40:51.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:40:51.360 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T08:40:51.360 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T08:40:51.360 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T08:40:51.363 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T08:40:51.363 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T08:41:21.746 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5842453448737408) > weight (0.25) 2021-02-20T08:41:21.747 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T08:41:21.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T08:41:21.748 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T08:41:21.748 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T08:41:21.749 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:41:21.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T08:41:21.745+0000 7f4c7b922700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 30730) UID: 1000 2021-02-20T08:41:21.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:41:27.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:41:27.851 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T08:41:27.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T08:41:27.851 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T08:41:27.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T08:41:27.854 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T08:41:54.365 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.26394412037078385) > weight (0.25) 2021-02-20T08:41:54.365 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3007382339665743) > weight (0.25) 2021-02-20T08:41:54.366 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.413264806467859) > weight (0.25) 2021-02-20T08:41:54.366 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T08:41:54.366 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T08:41:54.367 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T08:41:54.368 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T08:41:54.360+0000 7fb3ac279700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 27882) UID: 1000 2021-02-20T08:41:54.568 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:41:55.368 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T08:41:55.369 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:41:55.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:41:55.369 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T08:41:55.370 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T08:41:55.370 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T08:41:55.373 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T08:41:55.373 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T08:42:53.855 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.41142454032135356) > weight (0.25) 2021-02-20T08:42:53.856 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8280561321020861) > weight (0.25) 2021-02-20T08:42:53.856 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T08:42:53.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T08:42:53.858 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T08:42:53.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T08:42:53.855+0000 7f015dc6e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 31098) UID: 1000 2021-02-20T08:42:54.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:42:59.859 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T08:42:59.860 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:42:59.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:42:59.860 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T08:42:59.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T08:42:59.861 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T08:42:59.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T08:42:59.864 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T08:43:03.375 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.27705430933204644) > weight (0.25) 2021-02-20T08:43:03.375 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4667782667474287) > weight (0.25) 2021-02-20T08:43:03.375 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6960989662588792) > weight (0.25) 2021-02-20T08:43:03.376 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5849661958276271) > weight (0.25) 2021-02-20T08:43:03.376 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T08:44:27.865 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T08:44:27.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T08:44:27.868 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T08:44:27.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T08:44:27.866+0000 7fe141ef6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 31007) UID: 1000 2021-02-20T08:44:28.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:44:35.868 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T08:44:35.869 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:44:35.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:44:35.870 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T08:44:35.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T08:44:35.871 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T08:44:35.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T08:44:35.874 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T08:44:41.377 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6088992981401308) > weight (0.25) 2021-02-20T08:44:41.378 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9509539575552893) > weight (0.25) 2021-02-20T08:44:41.378 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9894449408841718) > weight (0.25) 2021-02-20T08:44:41.378 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.32155053779977294) > weight (0.25) 2021-02-20T08:44:41.379 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T08:45:44.875 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7618080183989393) > weight (0.25) 2021-02-20T08:45:44.876 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5571741386622235) > weight (0.25) 2021-02-20T08:45:44.876 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T08:45:44.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T08:45:44.877 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T08:45:44.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T08:45:44.872+0000 7f123da9f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 31463) UID: 1000 2021-02-20T08:45:45.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:45:53.878 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T08:45:53.879 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:45:53.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:45:53.880 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T08:45:53.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T08:45:53.881 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T08:45:53.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T08:45:53.884 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T08:46:23.379 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8901471099515251) > weight (0.25) 2021-02-20T08:46:23.380 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4829358597873842) > weight (0.25) 2021-02-20T08:46:23.380 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T08:46:23.381 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T08:46:23.381 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T08:46:23.382 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T08:46:23.380+0000 7f93873e7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 30915) UID: 1000 2021-02-20T08:46:23.582 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:46:27.382 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T08:46:27.383 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:46:27.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:46:27.384 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T08:46:27.384 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T08:46:27.384 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T08:46:27.387 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T08:46:27.387 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T08:47:43.886 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5603845715092619) > weight (0.25) 2021-02-20T08:47:43.886 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T08:47:43.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T08:47:43.887 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T08:47:43.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T08:47:43.886+0000 7f0bfb4f4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 31281) UID: 1000 2021-02-20T08:47:44.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:47:47.388 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.974708134346909) > weight (0.25) 2021-02-20T08:47:47.388 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8061218571491738) > weight (0.25) 2021-02-20T08:47:47.389 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5582165804188348) > weight (0.25) 2021-02-20T08:47:47.389 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T08:47:47.389 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T08:47:47.390 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T08:47:47.391 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T08:47:47.386+0000 7fcb99d1e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 31372) UID: 1000 2021-02-20T08:47:47.591 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:47:49.391 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T08:47:49.391 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:47:49.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:47:49.392 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T08:47:49.392 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T08:47:49.393 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T08:47:49.395 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T08:47:49.396 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T08:47:52.888 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T08:47:52.889 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:47:52.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:47:52.890 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T08:47:52.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T08:47:52.890 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T08:47:52.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T08:47:52.893 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T08:49:25.397 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8171398256888036) > weight (0.25) 2021-02-20T08:49:25.398 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T08:49:25.398 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T08:49:25.399 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T08:49:25.400 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T08:49:25.393+0000 7f2f1ea90700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 31190) UID: 1000 2021-02-20T08:49:25.600 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:49:30.400 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T08:49:30.400 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:49:30.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:49:30.401 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T08:49:30.401 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T08:49:30.402 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T08:49:30.405 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T08:49:30.405 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T08:49:47.894 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T08:49:47.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T08:49:47.895 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T08:49:47.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T08:49:47.892+0000 7feeab12a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 31554) UID: 1000 2021-02-20T08:49:48.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:49:48.896 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T08:49:48.897 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:49:48.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:49:48.898 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T08:49:48.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T08:49:48.898 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T08:49:48.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T08:49:48.901 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T08:51:05.406 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6811612752914427) > weight (0.25) 2021-02-20T08:51:05.407 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T08:51:05.407 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T08:51:05.408 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T08:51:05.409 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T08:51:05.407+0000 7f684708e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 32013) UID: 1000 2021-02-20T08:51:05.609 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:51:09.409 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T08:51:09.409 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:51:09.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:51:09.410 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T08:51:09.410 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T08:51:09.411 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T08:51:09.413 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T08:51:09.414 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T08:51:25.902 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5005697681273397) > weight (0.25) 2021-02-20T08:51:25.903 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T08:51:25.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T08:51:25.904 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T08:51:25.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T08:51:25.903+0000 7f445a94a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 31921) UID: 1000 2021-02-20T08:51:26.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:51:32.905 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T08:51:32.905 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:51:32.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:51:32.906 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T08:51:32.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T08:51:32.907 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T08:51:32.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T08:51:32.910 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T08:52:16.415 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3921837121244792) > weight (0.25) 2021-02-20T08:52:16.417 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6845428137218118) > weight (0.25) 2021-02-20T08:52:16.418 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.28064114284798103) > weight (0.25) 2021-02-20T08:52:16.418 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.464681496567779) > weight (0.25) 2021-02-20T08:52:16.418 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T08:53:02.911 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4522650456221726) > weight (0.25) 2021-02-20T08:53:02.919 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T08:53:02.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T08:53:02.920 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T08:53:02.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T08:53:02.917+0000 7f5efcb80700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 32287) UID: 1000 2021-02-20T08:53:03.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:53:10.921 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T08:53:10.929 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:53:10.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:53:10.930 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T08:53:10.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T08:53:10.931 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T08:53:10.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T08:53:10.934 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T08:53:59.419 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5378517012902811) > weight (0.25) 2021-02-20T08:53:59.420 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.30768287867592214) > weight (0.25) 2021-02-20T08:53:59.420 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T08:53:59.420 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T08:53:59.421 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T08:53:59.422 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T08:53:59.420+0000 7f680defd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 31739) UID: 1000 2021-02-20T08:53:59.622 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:54:02.422 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T08:54:02.422 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:54:02.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:54:02.423 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T08:54:02.423 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T08:54:02.424 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T08:54:02.427 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T08:54:02.427 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T08:55:00.935 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7422311633550244) > weight (0.25) 2021-02-20T08:55:00.938 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8631477546036859) > weight (0.25) 2021-02-20T08:55:00.938 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T08:55:00.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T08:55:00.939 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T08:55:00.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T08:55:00.939+0000 7f5e5c742700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 31648) UID: 1000 2021-02-20T08:55:01.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:55:01.940 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T08:55:01.941 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:55:01.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:55:01.942 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T08:55:01.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T08:55:01.942 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T08:55:01.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T08:55:01.946 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T08:55:13.428 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6659992212158368) > weight (0.25) 2021-02-20T08:55:13.441 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T08:55:13.442 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T08:55:13.442 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T08:55:13.443 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T08:55:13.439+0000 7f2c1453a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 32196) UID: 1000 2021-02-20T08:55:13.644 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:55:14.444 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T08:55:14.444 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:55:14.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:55:14.445 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T08:55:14.445 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T08:55:14.446 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T08:55:14.448 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T08:55:14.449 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T08:56:26.947 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T08:56:26.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T08:56:26.949 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T08:56:26.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T08:56:26.950+0000 7f4bdba36700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 32104) UID: 1000 2021-02-20T08:56:27.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:56:35.950 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T08:56:35.950 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:56:35.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:56:35.951 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T08:56:35.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T08:56:35.952 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T08:56:35.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T08:56:35.955 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T08:56:37.450 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5621060804590138) > weight (0.25) 2021-02-20T08:56:37.450 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T08:56:37.451 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T08:56:37.451 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T08:56:37.452 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T08:56:37.449+0000 7f59a8cfe700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 32654) UID: 1000 2021-02-20T08:56:37.652 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:56:38.451 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T08:56:38.452 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:56:38.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:56:38.452 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T08:56:38.453 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T08:56:38.453 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T08:56:38.456 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T08:56:38.456 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T08:58:09.457 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T08:58:09.494 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T08:58:09.494 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T08:58:09.495 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T08:58:09.492+0000 7f6b792a3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 29023) UID: 1000 2021-02-20T08:58:09.896 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:58:11.956 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T08:58:11.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T08:58:11.962 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T08:58:11.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T08:58:11.960+0000 7fb60ba7d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 32745) UID: 1000 2021-02-20T08:58:12.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:58:13.710 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T08:58:13.708+0000 7ff1a0030700 -1 rbd::mirror::ImageReplayer: 0x55b7feca5900 [3/3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd] start: already running 2021-02-20T08:58:16.495 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T08:58:16.502 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:58:16.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:58:16.503 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T08:58:16.504 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T08:58:16.504 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T08:58:16.507 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T08:58:16.508 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T08:58:17.962 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T08:58:17.971 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:58:17.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:58:17.972 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T08:58:17.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T08:58:17.974 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T08:58:17.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T08:58:17.977 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T09:00:14.510 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.41158782448198383) > weight (0.25) 2021-02-20T09:00:14.512 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T09:00:14.513 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T09:00:14.514 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T09:00:14.515 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T09:00:14.510+0000 7f20c8cb5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 32836) UID: 1000 2021-02-20T09:00:14.715 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:00:16.979 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8000973641320794) > weight (0.25) 2021-02-20T09:00:16.980 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:00:16.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:00:16.980 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T09:00:16.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T09:00:16.978+0000 7f497824e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 32378) UID: 1000 2021-02-20T09:00:17.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:00:17.981 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:00:17.982 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:00:17.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:00:17.983 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:00:17.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:00:17.983 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:00:17.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:00:17.987 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T09:00:21.515 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T09:00:21.516 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:00:21.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:00:21.517 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T09:00:21.517 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T09:00:21.517 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T09:00:21.520 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T09:00:21.520 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T09:01:18.988 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.837458300536246) > weight (0.25) 2021-02-20T09:01:18.997 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:01:18.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:01:18.999 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T09:01:19.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T09:01:18.997+0000 7f47f54ce700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 33112) UID: 1000 2021-02-20T09:01:19.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:01:26.000 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:01:26.010 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:01:26.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:01:26.011 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:01:26.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:01:26.012 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:01:26.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:01:26.015 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T09:01:53.521 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4692741631848415) > weight (0.25) 2021-02-20T09:01:53.531 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4344750342180891) > weight (0.25) 2021-02-20T09:01:53.531 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9739418729013244) > weight (0.25) 2021-02-20T09:01:53.531 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T09:01:53.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T09:01:53.532 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T09:01:53.533 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T09:01:53.532+0000 7ff1a1032700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 31830) UID: 1000 2021-02-20T09:01:53.934 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:02:00.533 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T09:02:00.544 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:02:00.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:02:00.545 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T09:02:00.545 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T09:02:00.547 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T09:02:00.551 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T09:02:00.551 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T09:02:39.016 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9565768334113384) > weight (0.25) 2021-02-20T09:02:39.017 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9522106989118237) > weight (0.25) 2021-02-20T09:02:39.017 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4509871650715197) > weight (0.25) 2021-02-20T09:02:39.017 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5770398407286965) > weight (0.25) 2021-02-20T09:02:39.018 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T09:03:45.553 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7930705806002531) > weight (0.25) 2021-02-20T09:03:45.565 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.30876807714406296) > weight (0.25) 2021-02-20T09:03:45.566 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9638356509169832) > weight (0.25) 2021-02-20T09:03:45.566 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8857348810553367) > weight (0.25) 2021-02-20T09:03:45.567 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T09:04:16.018 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.34086323657539896) > weight (0.25) 2021-02-20T09:04:16.084 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.41808238630564487) > weight (0.25) 2021-02-20T09:04:16.085 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5093094180024532) > weight (0.25) 2021-02-20T09:04:16.086 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6590494532730264) > weight (0.25) 2021-02-20T09:04:16.086 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T09:05:09.569 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7186849623646471) > weight (0.25) 2021-02-20T09:05:09.575 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7493716301452474) > weight (0.25) 2021-02-20T09:05:09.576 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7463609587824013) > weight (0.25) 2021-02-20T09:05:09.576 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7541463016432927) > weight (0.25) 2021-02-20T09:05:09.576 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T09:05:49.087 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T09:05:49.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T09:05:49.089 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T09:05:49.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T09:05:49.088+0000 7f9f68277700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 33019) UID: 1000 2021-02-20T09:05:49.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:05:52.090 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T09:05:52.091 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:05:52.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:05:52.091 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T09:05:52.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T09:05:52.092 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T09:05:52.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T09:05:52.095 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T09:06:59.578 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.940939361247095) > weight (0.25) 2021-02-20T09:06:59.579 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T09:06:59.580 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T09:06:59.581 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T09:06:59.582 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T09:06:59.579+0000 7f0fff281700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 33203) UID: 1000 2021-02-20T09:06:59.782 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:07:06.582 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T09:07:06.583 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:07:06.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:07:06.584 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T09:07:06.584 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T09:07:06.585 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T09:07:06.588 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T09:07:06.588 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T09:07:46.096 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8770743704346944) > weight (0.25) 2021-02-20T09:07:46.097 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2903922886008474) > weight (0.25) 2021-02-20T09:07:46.098 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T09:07:46.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T09:07:46.100 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T09:07:46.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T09:07:46.094+0000 7f45f7503700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 32563) UID: 1000 2021-02-20T09:07:46.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:07:54.101 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T09:07:54.102 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:07:54.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:07:54.103 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T09:07:54.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T09:07:54.103 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T09:07:54.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T09:07:54.106 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T09:08:00.835 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T09:08:00.829+0000 7fb533c45700 -1 reset not still connected to 0x558b94b90be0 2021-02-20T09:08:30.827 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T09:08:30.821+0000 7fb533c45700 -1 reset not still connected to 0x558b84afaff0 2021-02-20T09:08:30.827 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T09:08:30.821+0000 7fb533c45700 -1 reset not still connected to 0x558b8917c1a0 2021-02-20T09:08:30.828 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T09:08:30.821+0000 7fb533c45700 -1 reset not still connected to 0x558b94b90be0 2021-02-20T09:08:59.590 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4936520092316794) > weight (0.25) 2021-02-20T09:08:59.599 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T09:08:59.600 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T09:08:59.601 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T09:08:59.602 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T09:08:59.597+0000 7ff602918700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 33571) UID: 1000 2021-02-20T09:08:59.803 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:09:05.603 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T09:09:05.603 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:09:05.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:09:05.604 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T09:09:05.604 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T09:09:05.605 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T09:09:05.608 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T09:09:05.608 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T09:09:46.108 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8366517872271486) > weight (0.25) 2021-02-20T09:09:46.188 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:09:46.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:09:46.190 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T09:09:46.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T09:09:46.188+0000 7f11e03cd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 33294) UID: 1000 2021-02-20T09:09:46.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:09:47.191 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:09:47.191 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:09:47.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:09:47.192 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:09:47.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:09:47.193 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:09:47.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:09:47.196 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T09:10:38.610 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8254007029667509) > weight (0.25) 2021-02-20T09:10:38.611 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6801496680648397) > weight (0.25) 2021-02-20T09:10:38.611 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T09:10:38.612 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T09:10:38.612 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T09:10:38.613 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T09:10:38.613 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:10:38.614 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T09:10:38.611+0000 7f060feeb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 32469) UID: 1000 2021-02-20T09:10:38.814 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:10:44.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:10:44.727 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T09:10:44.727 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T09:10:44.728 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T09:10:44.730 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T09:10:44.731 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T09:11:43.197 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6558730462228469) > weight (0.25) 2021-02-20T09:11:43.347 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.39882286933849853) > weight (0.25) 2021-02-20T09:11:43.348 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4430515533581775) > weight (0.25) 2021-02-20T09:11:43.348 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T09:11:43.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T09:11:43.349 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T09:11:43.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T09:11:43.350+0000 7f3d9e9b4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 21321) UID: 1000 2021-02-20T09:11:43.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:11:44.349 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T09:11:44.350 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:11:44.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:11:44.351 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T09:11:44.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T09:11:44.351 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T09:11:44.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T09:11:44.354 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T09:11:46.731 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6502571338542245) > weight (0.25) 2021-02-20T09:11:46.737 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T09:11:46.737 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T09:11:46.738 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T09:11:46.738 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T09:11:46.734+0000 7f3df8298700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 33753) UID: 1000 2021-02-20T09:11:46.939 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:11:51.738 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T09:11:51.745 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:11:51.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:11:51.746 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T09:11:51.746 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T09:11:51.747 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T09:11:51.749 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T09:11:51.750 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T09:12:54.355 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8472426104782431) > weight (0.25) 2021-02-20T09:12:54.361 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8332050615907134) > weight (0.25) 2021-02-20T09:12:54.362 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5096966192327175) > weight (0.25) 2021-02-20T09:12:54.362 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9878539943884956) > weight (0.25) 2021-02-20T09:12:54.362 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T09:13:49.750 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T09:13:49.808 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T09:13:49.809 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T09:13:49.810 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T09:13:49.808+0000 7faac2b31700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 32928) UID: 1000 2021-02-20T09:13:50.010 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:13:57.810 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T09:13:57.814 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:13:57.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:13:57.814 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T09:13:57.815 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T09:13:57.815 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T09:13:57.820 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T09:13:57.820 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T09:14:53.363 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T09:14:53.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T09:14:53.368 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T09:14:53.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T09:14:53.363+0000 7f97d1aa6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 33480) UID: 1000 2021-02-20T09:14:53.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:14:54.368 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T09:14:54.369 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:14:54.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:14:54.370 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T09:14:54.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T09:14:54.370 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T09:14:54.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T09:14:54.374 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T09:15:51.821 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.745141795898481) > weight (0.25) 2021-02-20T09:15:51.856 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T09:15:51.857 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T09:15:51.858 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T09:15:51.858 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T09:15:51.854+0000 7fe0a4a73700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 34128) UID: 1000 2021-02-20T09:15:52.059 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:15:53.859 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T09:15:53.859 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:15:53.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:15:53.860 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T09:15:53.861 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T09:15:53.861 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T09:15:53.864 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T09:15:53.865 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T09:16:07.375 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6296785469268067) > weight (0.25) 2021-02-20T09:16:07.385 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5052037999839313) > weight (0.25) 2021-02-20T09:16:07.385 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T09:16:07.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T09:16:07.386 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T09:16:07.387 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T09:16:07.387 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:16:07.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T09:16:07.385+0000 7f19b8bb7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 33662) UID: 1000 2021-02-20T09:16:07.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:16:13.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:16:13.495 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T09:16:13.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T09:16:13.496 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T09:16:13.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T09:16:13.502 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T09:17:09.866 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.30376052549377464) > weight (0.25) 2021-02-20T09:17:09.876 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.37239798103660104) > weight (0.25) 2021-02-20T09:17:09.876 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5895645493291111) > weight (0.25) 2021-02-20T09:17:09.876 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5901483027860019) > weight (0.25) 2021-02-20T09:17:09.877 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T09:17:56.503 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4966292771950849) > weight (0.25) 2021-02-20T09:17:56.870 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3728490783244285) > weight (0.25) 2021-02-20T09:17:56.871 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.37232001223674926) > weight (0.25) 2021-02-20T09:17:56.871 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5578391960237404) > weight (0.25) 2021-02-20T09:17:56.871 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T09:19:00.877 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6295170078292943) > weight (0.25) 2021-02-20T09:19:00.878 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4063041009157705) > weight (0.25) 2021-02-20T09:19:00.879 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T09:19:00.880 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T09:19:00.880 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T09:19:00.881 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T09:19:00.874+0000 7f2d4a22f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 33937) UID: 1000 2021-02-20T09:19:01.081 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:19:06.881 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T09:19:06.882 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:19:06.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:19:06.883 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T09:19:06.883 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T09:19:06.884 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T09:19:06.888 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T09:19:06.888 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T09:19:28.872 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8598156529521168) > weight (0.25) 2021-02-20T09:19:28.873 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8951315807232332) > weight (0.25) 2021-02-20T09:19:28.873 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9032000014822589) > weight (0.25) 2021-02-20T09:19:28.873 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6131058500255464) > weight (0.25) 2021-02-20T09:19:28.873 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T09:20:25.890 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3923668926118028) > weight (0.25) 2021-02-20T09:20:25.899 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8081263784183285) > weight (0.25) 2021-02-20T09:20:25.899 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T09:20:25.900 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T09:20:25.901 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T09:20:25.902 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T09:20:25.897+0000 7f8f2301a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 34607) UID: 1000 2021-02-20T09:20:26.102 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:20:29.902 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T09:20:29.909 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:20:29.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:20:29.910 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T09:20:29.911 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T09:20:29.911 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T09:20:29.914 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T09:20:29.914 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T09:21:02.875 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7386881267133801) > weight (0.25) 2021-02-20T09:21:02.875 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.646966783251126) > weight (0.25) 2021-02-20T09:21:02.876 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7602036824754815) > weight (0.25) 2021-02-20T09:21:02.876 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7176676694125164) > weight (0.25) 2021-02-20T09:21:02.876 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T09:22:24.915 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8836164882731543) > weight (0.25) 2021-02-20T09:22:24.916 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3600659676042194) > weight (0.25) 2021-02-20T09:22:24.917 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T09:22:24.917 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T09:22:24.918 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T09:22:24.919 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T09:22:24.915+0000 7f5a96758700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 34699) UID: 1000 2021-02-20T09:22:25.119 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:22:27.919 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T09:22:27.919 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:22:27.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:22:27.920 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T09:22:27.920 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T09:22:27.921 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T09:22:27.925 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T09:22:27.925 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T09:22:28.877 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.33688705559196286) > weight (0.25) 2021-02-20T09:22:28.878 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.453726125242918) > weight (0.25) 2021-02-20T09:22:28.878 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.92666343817228) > weight (0.25) 2021-02-20T09:22:28.878 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6395340578169282) > weight (0.25) 2021-02-20T09:22:28.879 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T09:23:46.926 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T09:23:46.931 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T09:23:46.932 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T09:23:46.933 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T09:23:46.925+0000 7ff0ea22e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 34219) UID: 1000 2021-02-20T09:23:47.133 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:23:49.880 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.60592452007305) > weight (0.25) 2021-02-20T09:23:49.880 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:23:49.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:23:49.881 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T09:23:49.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T09:23:49.877+0000 7fb1acd2d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 33844) UID: 1000 2021-02-20T09:23:49.933 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T09:23:49.934 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:23:49.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:23:49.935 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T09:23:49.935 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T09:23:49.935 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T09:23:49.938 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T09:23:49.938 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T09:23:50.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:23:53.882 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:23:53.882 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:23:53.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:23:53.883 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:23:53.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:23:53.884 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:23:53.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:23:53.887 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T09:25:03.941 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8262560404938994) > weight (0.25) 2021-02-20T09:25:03.950 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5988851541620565) > weight (0.25) 2021-02-20T09:25:03.951 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8173994639856325) > weight (0.25) 2021-02-20T09:25:03.951 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9357512371180149) > weight (0.25) 2021-02-20T09:25:03.951 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T09:25:41.889 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2619526164858329) > weight (0.25) 2021-02-20T09:25:41.895 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:25:41.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:25:41.897 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T09:25:41.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T09:25:41.896+0000 7fca050cf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 34972) UID: 1000 2021-02-20T09:25:42.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:25:43.898 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:25:43.900 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:25:43.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:25:43.903 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:25:43.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:25:43.905 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:25:43.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:25:43.911 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T09:26:53.912 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T09:26:53.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T09:26:53.923 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T09:26:53.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T09:26:53.918+0000 7fee59efa700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 34312) UID: 1000 2021-02-20T09:26:54.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:26:56.923 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T09:26:56.932 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:26:56.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:26:56.933 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T09:26:56.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T09:26:56.934 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T09:26:56.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T09:26:56.937 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T09:26:57.952 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5047542276237211) > weight (0.25) 2021-02-20T09:26:57.952 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.46617853339498694) > weight (0.25) 2021-02-20T09:26:57.953 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T09:26:57.953 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T09:26:57.953 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T09:26:57.954 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T09:26:57.950+0000 7f3beebe9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 34790) UID: 1000 2021-02-20T09:26:58.155 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:27:05.954 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T09:27:05.955 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:27:05.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:27:05.955 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T09:27:05.956 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T09:27:05.956 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T09:27:05.961 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T09:27:05.961 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T09:28:10.964 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.32020043913132923) > weight (0.25) 2021-02-20T09:28:11.089 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T09:28:11.091 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T09:28:11.091 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T09:28:11.092 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T09:28:11.089+0000 7fb430b17700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 34414) UID: 1000 2021-02-20T09:28:11.292 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:28:13.092 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T09:28:13.092 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:28:13.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:28:13.093 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T09:28:13.094 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T09:28:13.095 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T09:28:13.097 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T09:28:13.098 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T09:28:39.939 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3745389518399296) > weight (0.25) 2021-02-20T09:28:39.945 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5572321600705553) > weight (0.25) 2021-02-20T09:28:39.945 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.27015243192739913) > weight (0.25) 2021-02-20T09:28:39.945 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T09:28:39.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T09:28:39.946 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T09:28:39.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T09:28:39.945+0000 7fbec6b7d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 34029) UID: 1000 2021-02-20T09:28:40.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:28:45.947 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T09:28:45.948 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:28:45.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:28:45.949 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T09:28:45.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T09:28:45.950 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T09:28:45.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T09:28:45.952 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T09:29:30.098 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6860410287535931) > weight (0.25) 2021-02-20T09:29:30.099 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5885803582119936) > weight (0.25) 2021-02-20T09:29:30.099 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.41774608891285725) > weight (0.25) 2021-02-20T09:29:30.099 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T09:29:30.100 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T09:29:30.101 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T09:29:30.101 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T09:29:30.100+0000 7f38cd024700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 33385) UID: 1000 2021-02-20T09:29:30.502 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:29:34.101 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T09:29:34.102 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:29:34.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:29:34.103 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T09:29:34.103 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T09:29:34.104 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T09:29:34.107 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T09:29:34.107 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T09:30:41.954 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2894024194167364) > weight (0.25) 2021-02-20T09:30:41.960 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:30:41.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:30:41.961 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T09:30:41.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T09:30:41.958+0000 7f6390549700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 35067) UID: 1000 2021-02-20T09:30:42.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:30:42.962 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:30:42.963 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:30:42.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:30:42.963 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:30:42.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:30:42.964 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:30:42.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:30:42.967 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T09:30:48.108 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T09:30:48.109 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T09:30:48.109 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T09:30:48.110 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T09:30:48.106+0000 7fb222e31700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 34881) UID: 1000 2021-02-20T09:30:48.511 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:30:56.110 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T09:30:56.111 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:30:56.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:30:56.112 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T09:30:56.112 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T09:30:56.113 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T09:30:56.115 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T09:30:56.116 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T09:32:04.117 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T09:32:04.133 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T09:32:04.134 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T09:32:04.135 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T09:32:04.129+0000 7f651458a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 35714) UID: 1000 2021-02-20T09:32:04.335 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:32:04.969 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.913877068944413) > weight (0.25) 2021-02-20T09:32:04.969 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:32:04.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:32:04.970 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T09:32:04.970 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:32:04.970 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:32:04.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T09:32:04.965+0000 7fed05d10700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 35623) UID: 1000 2021-02-20T09:32:05.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:32:08.136 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T09:32:08.136 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:32:08.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:32:08.137 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T09:32:08.137 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T09:32:08.138 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T09:32:08.141 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T09:32:08.141 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T09:32:11.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:32:11.073 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:32:11.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:32:11.074 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:32:11.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:32:11.081 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T09:33:43.083 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T09:33:43.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T09:33:43.086 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T09:33:43.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T09:33:43.083+0000 7f7f3618a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 35158) UID: 1000 2021-02-20T09:33:43.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:33:44.142 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4239827605394445) > weight (0.25) 2021-02-20T09:33:44.143 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T09:33:44.143 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T09:33:44.145 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T09:33:44.146 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T09:33:44.139+0000 7f3e96ae0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 35348) UID: 1000 2021-02-20T09:33:44.546 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:33:48.086 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T09:33:48.087 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:33:48.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:33:48.088 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T09:33:48.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T09:33:48.088 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T09:33:48.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T09:33:48.091 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T09:33:52.146 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T09:33:52.147 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:33:52.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:33:52.148 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T09:33:52.148 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T09:33:52.148 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T09:33:52.151 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T09:33:52.151 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T09:35:23.093 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T09:35:23.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T09:35:23.095 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T09:35:23.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T09:35:23.090+0000 7f9d9563d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 35988) UID: 1000 2021-02-20T09:35:23.151 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8180516347081125) > weight (0.25) 2021-02-20T09:35:23.152 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.44596241116946134) > weight (0.25) 2021-02-20T09:35:23.152 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7874522257456218) > weight (0.25) 2021-02-20T09:35:23.152 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4696356521957673) > weight (0.25) 2021-02-20T09:35:23.153 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T09:35:23.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:35:24.096 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T09:35:24.097 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:35:24.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:35:24.097 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T09:35:24.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T09:35:24.098 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T09:35:24.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T09:35:24.101 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T09:36:48.154 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.27353991949330203) > weight (0.25) 2021-02-20T09:36:48.197 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T09:36:48.198 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T09:36:48.198 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T09:36:48.199 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T09:36:48.196+0000 7fc23d56b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 36079) UID: 1000 2021-02-20T09:36:48.400 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:36:54.200 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T09:36:54.200 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:36:54.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:36:54.201 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T09:36:54.201 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T09:36:54.202 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T09:36:54.204 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T09:36:54.205 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T09:37:16.103 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T09:37:16.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T09:37:16.105 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T09:37:16.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T09:37:16.104+0000 7fdfd7565700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 36173) UID: 1000 2021-02-20T09:37:16.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:37:18.105 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T09:37:18.106 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:37:18.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:37:18.107 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T09:37:18.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T09:37:18.107 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T09:37:18.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T09:37:18.110 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T09:38:02.205 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T09:38:02.214 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T09:38:02.215 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T09:38:02.216 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T09:38:02.211+0000 7fb076ef5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 35805) UID: 1000 2021-02-20T09:38:02.416 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:38:10.216 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T09:38:10.278 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:38:10.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:38:10.279 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T09:38:10.280 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T09:38:10.280 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T09:38:10.283 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T09:38:10.284 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T09:38:36.111 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.34422665592574975) > weight (0.25) 2021-02-20T09:38:36.112 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:38:36.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:38:36.113 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T09:38:36.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T09:38:36.110+0000 7f498597c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 35896) UID: 1000 2021-02-20T09:38:36.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:38:43.113 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:38:43.114 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:38:43.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:38:43.115 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:38:43.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:38:43.115 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:38:43.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:38:43.122 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T09:39:53.285 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6878586247418182) > weight (0.25) 2021-02-20T09:39:53.293 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2819927501260884) > weight (0.25) 2021-02-20T09:39:53.293 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T09:39:53.294 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T09:39:53.295 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T09:39:53.296 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T09:39:53.289+0000 7ffba55b2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 35257) UID: 1000 2021-02-20T09:39:53.695 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:39:57.296 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T09:39:57.302 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:39:57.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:39:57.303 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T09:39:57.303 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T09:39:57.304 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T09:39:57.306 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T09:39:57.307 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T09:40:32.124 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8415994154932707) > weight (0.25) 2021-02-20T09:40:32.134 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:40:32.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:40:32.136 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T09:40:32.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T09:40:32.136+0000 7ff7ede60700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 36537) UID: 1000 2021-02-20T09:40:32.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:40:41.136 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:40:41.141 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:40:41.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:40:41.142 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:40:41.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:40:41.143 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:40:41.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:40:41.146 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T09:41:42.308 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9848079246538658) > weight (0.25) 2021-02-20T09:41:42.315 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3778144183548887) > weight (0.25) 2021-02-20T09:41:42.316 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T09:41:42.316 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T09:41:42.317 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T09:41:42.318 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T09:41:42.315+0000 7efd715d7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 36629) UID: 1000 2021-02-20T09:41:42.519 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:41:48.148 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.782734131330592) > weight (0.25) 2021-02-20T09:41:48.149 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.43135220983952227) > weight (0.25) 2021-02-20T09:41:48.149 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.35617571219003963) > weight (0.25) 2021-02-20T09:41:48.149 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.36324563606955096) > weight (0.25) 2021-02-20T09:41:48.150 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T09:41:49.318 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T09:41:49.318 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:41:49.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:41:49.319 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T09:41:49.319 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T09:41:49.320 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T09:41:49.323 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T09:41:49.323 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T09:43:31.324 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9729319491495994) > weight (0.25) 2021-02-20T09:43:31.332 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5846704966862247) > weight (0.25) 2021-02-20T09:43:31.332 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T09:43:31.334 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T09:43:31.334 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T09:43:31.335 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T09:43:31.329+0000 7f3d4f3a5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 36812) UID: 1000 2021-02-20T09:43:31.535 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:43:33.335 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T09:43:33.336 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:43:33.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:43:33.337 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T09:43:33.337 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T09:43:33.337 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T09:43:33.340 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T09:43:33.341 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T09:43:36.150 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.46847111978776923) > weight (0.25) 2021-02-20T09:43:36.151 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6337122313256586) > weight (0.25) 2021-02-20T09:43:36.151 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.26228153036104374) > weight (0.25) 2021-02-20T09:43:36.151 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3356599867610114) > weight (0.25) 2021-02-20T09:43:36.151 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T09:44:59.342 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3706039166602355) > weight (0.25) 2021-02-20T09:44:59.351 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.868436069680128) > weight (0.25) 2021-02-20T09:44:59.352 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.44694423716318976) > weight (0.25) 2021-02-20T09:44:59.352 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9461052264398817) > weight (0.25) 2021-02-20T09:44:59.352 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T09:45:06.152 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8559474547121112) > weight (0.25) 2021-02-20T09:45:06.159 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.36597372738526956) > weight (0.25) 2021-02-20T09:45:06.159 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T09:45:06.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T09:45:06.161 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T09:45:06.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T09:45:06.160+0000 7f0ca88b1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 34513) UID: 1000 2021-02-20T09:45:06.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:45:14.162 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T09:45:14.163 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:45:14.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:45:14.164 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T09:45:14.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T09:45:14.164 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T09:45:14.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T09:45:14.167 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T09:46:17.169 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.35422096585427565) > weight (0.25) 2021-02-20T09:46:17.175 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.34665172452415594) > weight (0.25) 2021-02-20T09:46:17.175 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5818830276918204) > weight (0.25) 2021-02-20T09:46:17.175 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T09:46:17.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T09:46:17.176 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T09:46:17.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T09:46:17.171+0000 7f356e597700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 35439) UID: 1000 2021-02-20T09:46:17.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:46:24.484 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T09:46:24.478+0000 7fb533c45700 -1 reset not still connected to 0x558b95d3f8e0 2021-02-20T09:46:26.177 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T09:46:26.179 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:46:26.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:46:26.180 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T09:46:26.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T09:46:26.181 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T09:46:26.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T09:46:26.185 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T09:46:30.353 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7265922388973529) > weight (0.25) 2021-02-20T09:46:30.354 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7186818506670294) > weight (0.25) 2021-02-20T09:46:30.354 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.40700585727065475) > weight (0.25) 2021-02-20T09:46:30.354 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8266524210768971) > weight (0.25) 2021-02-20T09:46:30.354 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T09:47:49.186 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9676368945506597) > weight (0.25) 2021-02-20T09:47:49.194 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:47:49.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:47:49.196 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T09:47:49.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T09:47:49.189+0000 7fb0b8f34700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 36721) UID: 1000 2021-02-20T09:47:49.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:47:51.197 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:47:51.197 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:47:51.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:47:51.198 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:47:51.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:47:51.199 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:47:51.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:47:51.202 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T09:48:03.355 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T09:48:03.363 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T09:48:03.363 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T09:48:03.364 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T09:48:03.357+0000 7efdc1688700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 36446) UID: 1000 2021-02-20T09:48:03.565 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:48:08.365 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T09:48:08.365 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:48:08.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:48:08.366 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T09:48:08.366 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T09:48:08.367 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T09:48:08.370 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T09:48:08.370 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T09:49:25.210 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5868415247859622) > weight (0.25) 2021-02-20T09:49:25.211 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:49:25.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:49:25.213 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T09:49:25.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T09:49:25.211+0000 7f61658d4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 37267) UID: 1000 2021-02-20T09:49:25.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:49:31.370 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T09:49:31.371 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T09:49:31.372 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T09:49:31.375 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T09:49:31.371+0000 7f9702f3d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 37359) UID: 1000 2021-02-20T09:49:31.574 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:49:33.214 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:49:33.215 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:49:33.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:49:33.216 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:49:33.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:49:33.216 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:49:33.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:49:33.220 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T09:49:38.372 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T09:49:38.373 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:49:38.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:49:38.374 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T09:49:38.374 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T09:49:38.374 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T09:49:38.377 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T09:49:38.377 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T09:50:49.379 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.486922850613959) > weight (0.25) 2021-02-20T09:50:49.387 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3108655223848985) > weight (0.25) 2021-02-20T09:50:49.387 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6743485640469706) > weight (0.25) 2021-02-20T09:50:49.387 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.41916261236069263) > weight (0.25) 2021-02-20T09:50:49.388 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T09:51:27.221 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3018934105571144) > weight (0.25) 2021-02-20T09:51:27.222 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7610741985841081) > weight (0.25) 2021-02-20T09:51:27.222 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.48469217273135556) > weight (0.25) 2021-02-20T09:51:27.222 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.740096340063721) > weight (0.25) 2021-02-20T09:51:27.222 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T09:52:05.388 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8087411758901213) > weight (0.25) 2021-02-20T09:52:05.396 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.25699659745249226) > weight (0.25) 2021-02-20T09:52:05.396 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T09:52:05.397 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T09:52:05.397 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T09:52:05.398 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T09:52:05.397+0000 7efde2995700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 36982) UID: 1000 2021-02-20T09:52:05.599 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:52:11.398 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T09:52:11.398 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:52:11.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:52:11.399 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T09:52:11.399 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T09:52:11.400 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T09:52:11.403 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T09:52:11.403 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T09:52:51.223 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6851708493971406) > weight (0.25) 2021-02-20T09:52:51.261 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6547620175569875) > weight (0.25) 2021-02-20T09:52:51.262 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9301425283421086) > weight (0.25) 2021-02-20T09:52:51.262 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9554205979645528) > weight (0.25) 2021-02-20T09:52:51.262 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T09:54:03.405 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5542972165437897) > weight (0.25) 2021-02-20T09:54:03.413 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.34437492049660046) > weight (0.25) 2021-02-20T09:54:03.413 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5480543683393615) > weight (0.25) 2021-02-20T09:54:03.413 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7344879959124816) > weight (0.25) 2021-02-20T09:54:03.414 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T09:54:49.264 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8632977997404552) > weight (0.25) 2021-02-20T09:54:49.273 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9303391819230548) > weight (0.25) 2021-02-20T09:54:49.273 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2960900998316155) > weight (0.25) 2021-02-20T09:54:49.274 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T09:54:49.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T09:54:49.276 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T09:54:49.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T09:54:49.274+0000 7f7212e7b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 37175) UID: 1000 2021-02-20T09:54:49.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:54:53.277 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T09:54:53.287 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:54:53.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:54:53.288 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T09:54:53.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T09:54:53.289 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T09:54:53.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T09:54:53.292 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T09:55:18.418 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T09:55:18.423 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T09:55:18.424 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T09:55:18.425 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T09:55:18.421+0000 7efc17a22700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 37541) UID: 1000 2021-02-20T09:55:18.626 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:55:23.425 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T09:55:23.432 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:55:23.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:55:23.433 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T09:55:23.433 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T09:55:23.434 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T09:55:23.449 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T09:55:23.449 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T09:56:07.294 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T09:56:07.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T09:56:07.326 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T09:56:07.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T09:56:07.325+0000 7f029a8ed700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 36355) UID: 1000 2021-02-20T09:56:07.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:56:10.327 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T09:56:10.333 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:56:10.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:56:10.334 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T09:56:10.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T09:56:10.335 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T09:56:10.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T09:56:10.338 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T09:56:30.450 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T09:56:30.462 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T09:56:30.462 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T09:56:30.463 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T09:56:30.460+0000 7f0578cc7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 37817) UID: 1000 2021-02-20T09:56:30.663 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:56:36.463 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T09:56:36.472 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:56:36.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:56:36.473 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T09:56:36.473 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T09:56:36.474 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T09:56:36.479 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T09:56:36.481 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T09:57:59.482 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T09:57:59.517 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T09:57:59.518 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T09:57:59.519 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T09:57:59.515+0000 7f31db6b8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 38007) UID: 1000 2021-02-20T09:57:59.719 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:58:02.339 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6260536574380678) > weight (0.25) 2021-02-20T09:58:02.346 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:58:02.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:58:02.347 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T09:58:02.347 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:58:02.348 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:58:02.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T09:58:02.343+0000 7f7d8dc24700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 37450) UID: 1000 2021-02-20T09:58:02.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:58:03.519 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T09:58:03.525 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:58:03.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:58:03.527 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T09:58:03.527 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T09:58:03.528 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T09:58:03.535 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T09:58:03.536 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T09:58:08.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:58:08.488 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:58:08.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:58:08.489 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:58:08.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:58:08.492 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T09:59:13.493 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T09:59:13.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T09:59:13.504 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T09:59:13.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T09:59:13.501+0000 7fc296a62700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 37908) UID: 1000 2021-02-20T09:59:13.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:59:17.505 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T09:59:17.505 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:59:17.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:59:17.506 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T09:59:17.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T09:59:17.507 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T09:59:17.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T09:59:17.510 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T09:59:32.537 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.987199301185162) > weight (0.25) 2021-02-20T09:59:32.546 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7845155714517447) > weight (0.25) 2021-02-20T09:59:32.546 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T09:59:32.547 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T09:59:32.547 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T09:59:32.549 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T09:59:32.545+0000 7f770cd2f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 37632) UID: 1000 2021-02-20T09:59:32.749 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:59:39.548 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T09:59:39.556 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:59:39.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:59:39.557 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T09:59:39.557 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T09:59:39.557 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T09:59:39.560 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T09:59:39.561 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T10:00:54.512 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:00:54.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:00:54.523 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T10:00:54.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T10:00:54.520+0000 7f9667940700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 38281) UID: 1000 2021-02-20T10:00:54.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:01:03.524 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:01:03.532 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:01:03.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:01:03.533 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:01:03.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:01:03.534 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:01:03.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:01:03.540 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T10:01:16.561 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8308453976805749) > weight (0.25) 2021-02-20T10:01:16.571 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2545068076288347) > weight (0.25) 2021-02-20T10:01:16.571 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6014240119616404) > weight (0.25) 2021-02-20T10:01:16.572 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7897562079559158) > weight (0.25) 2021-02-20T10:01:16.572 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T10:02:12.542 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:02:12.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:02:12.559 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T10:02:12.559 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:02:12.560 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:02:12.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T10:02:12.562+0000 7fcce4817700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 38464) UID: 1000 2021-02-20T10:02:12.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:02:18.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:02:18.668 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:02:18.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:02:18.669 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:02:18.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:02:18.672 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T10:02:56.573 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.881138123021753) > weight (0.25) 2021-02-20T10:02:56.584 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5721389983189697) > weight (0.25) 2021-02-20T10:02:56.584 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.757063401159003) > weight (0.25) 2021-02-20T10:02:56.584 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7003316934838327) > weight (0.25) 2021-02-20T10:02:56.585 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T10:04:14.674 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:04:14.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:04:14.718 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T10:04:14.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T10:04:14.716+0000 7f55a11f6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 38555) UID: 1000 2021-02-20T10:04:14.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:04:16.719 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:04:16.720 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:04:16.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:04:16.721 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:04:16.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:04:16.721 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:04:16.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:04:16.725 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T10:04:46.585 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5684089540996103) > weight (0.25) 2021-02-20T10:04:46.596 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8475333670678631) > weight (0.25) 2021-02-20T10:04:46.597 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9232847947016535) > weight (0.25) 2021-02-20T10:04:46.597 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T10:04:46.598 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T10:04:46.598 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T10:04:46.599 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T10:04:46.596+0000 7f79b6dac700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 35530) UID: 1000 2021-02-20T10:04:46.800 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:04:49.600 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T10:04:49.600 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:04:49.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:04:49.601 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T10:04:49.601 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T10:04:49.601 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T10:04:49.604 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T10:04:49.605 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T10:05:55.725 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:05:55.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:05:55.734 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T10:05:55.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T10:05:55.731+0000 7f4bcc898700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 38648) UID: 1000 2021-02-20T10:05:55.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:06:02.735 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:06:02.743 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:06:02.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:06:02.745 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:06:02.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:06:02.746 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:06:02.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:06:02.760 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T10:06:16.606 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9646893089042612) > weight (0.25) 2021-02-20T10:06:16.613 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7681102090144429) > weight (0.25) 2021-02-20T10:06:16.613 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T10:06:16.614 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T10:06:16.614 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T10:06:16.615 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T10:06:16.614+0000 7f24cbc31700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 38372) UID: 1000 2021-02-20T10:06:16.815 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:06:24.615 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T10:06:24.616 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:06:24.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:06:24.617 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T10:06:24.617 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T10:06:24.617 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T10:06:24.620 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T10:06:24.620 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T10:07:47.762 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5771693303400929) > weight (0.25) 2021-02-20T10:07:47.846 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8957913258870319) > weight (0.25) 2021-02-20T10:07:47.847 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4415624161845346) > weight (0.25) 2021-02-20T10:07:47.847 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8086525616697977) > weight (0.25) 2021-02-20T10:07:47.847 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T10:08:17.622 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:08:17.632 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:08:17.633 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T10:08:17.636 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T10:08:17.632+0000 7f944a03e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 38099) UID: 1000 2021-02-20T10:08:17.834 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:08:25.634 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:08:25.641 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:08:25.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:08:25.642 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:08:25.643 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:08:25.643 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:08:25.646 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:08:25.646 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T10:09:09.849 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.40564932209076077) > weight (0.25) 2021-02-20T10:09:09.857 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.536679935860861) > weight (0.25) 2021-02-20T10:09:09.857 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T10:09:09.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T10:09:09.859 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T10:09:09.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:09:09.859+0000 7fa59a9d2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 37084) UID: 1000 2021-02-20T10:09:10.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:09:18.861 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T10:09:18.870 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:09:18.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:09:18.871 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T10:09:18.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T10:09:18.872 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T10:09:18.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T10:09:18.877 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T10:09:42.647 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.41259126621203035) > weight (0.25) 2021-02-20T10:09:42.648 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8970463951438667) > weight (0.25) 2021-02-20T10:09:42.648 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2848877524734861) > weight (0.25) 2021-02-20T10:09:42.649 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T10:09:42.650 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T10:09:42.650 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T10:09:42.651 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T10:09:42.651+0000 7f79e80fb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 38739) UID: 1000 2021-02-20T10:09:42.851 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:09:44.651 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T10:09:44.657 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:09:44.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:09:44.659 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T10:09:44.659 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T10:09:44.660 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T10:09:44.664 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T10:09:44.665 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T10:10:38.878 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:10:38.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:10:38.888 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T10:10:38.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T10:10:38.886+0000 7f45206c1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 38833) UID: 1000 2021-02-20T10:10:39.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:10:46.889 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:10:46.896 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:10:46.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:10:46.897 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:10:46.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:10:46.898 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:10:46.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:10:46.901 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T10:11:23.666 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:11:23.677 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:11:23.678 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T10:11:23.679 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T10:11:23.673+0000 7f5ec5fcb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 39016) UID: 1000 2021-02-20T10:11:23.878 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:11:31.680 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:11:31.695 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:11:31.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:11:31.696 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:11:31.696 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:11:31.697 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:11:31.701 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:11:31.702 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T10:12:41.903 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8639464306455726) > weight (0.25) 2021-02-20T10:12:41.908 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7312453418045278) > weight (0.25) 2021-02-20T10:12:41.908 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6606898551285) > weight (0.25) 2021-02-20T10:12:41.908 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T10:12:41.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T10:12:41.910 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T10:12:41.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T10:12:41.908+0000 7fb6c748c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 37723) UID: 1000 2021-02-20T10:12:42.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:12:50.910 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T10:12:50.911 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:12:50.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:12:50.912 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T10:12:50.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T10:12:50.913 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T10:12:50.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T10:12:50.919 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T10:13:23.704 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3131945721445145) > weight (0.25) 2021-02-20T10:13:23.711 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7470107205963494) > weight (0.25) 2021-02-20T10:13:23.712 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9409059487250815) > weight (0.25) 2021-02-20T10:13:23.713 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T10:13:23.714 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T10:13:23.715 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T10:13:23.720 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T10:13:23.715+0000 7f5b8dbac700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 39198) UID: 1000 2021-02-20T10:13:23.918 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:13:27.717 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T10:13:27.725 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:13:27.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:13:27.725 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T10:13:27.726 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T10:13:27.727 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T10:13:27.729 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T10:13:27.730 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T10:14:38.731 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8682436804215753) > weight (0.25) 2021-02-20T10:14:38.766 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T10:14:38.767 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T10:14:38.767 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T10:14:38.769 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T10:14:38.766+0000 7f1f67c74700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 36264) UID: 1000 2021-02-20T10:14:39.169 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:14:41.921 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7410444995876043) > weight (0.25) 2021-02-20T10:14:41.929 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7349252210666228) > weight (0.25) 2021-02-20T10:14:41.929 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9161922955541317) > weight (0.25) 2021-02-20T10:14:41.930 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9156447482058783) > weight (0.25) 2021-02-20T10:14:41.930 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T10:14:43.768 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T10:14:43.768 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:14:43.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:14:43.769 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T10:14:43.769 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T10:14:43.770 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T10:14:43.779 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T10:14:43.780 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T10:16:00.931 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5305428411457602) > weight (0.25) 2021-02-20T10:16:00.938 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3518339892428808) > weight (0.25) 2021-02-20T10:16:00.938 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T10:16:00.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T10:16:00.940 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T10:16:00.940 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T10:16:00.940 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:16:00.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:16:00.936+0000 7fe2e53ef700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 39107) UID: 1000 2021-02-20T10:16:01.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:16:07.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:16:07.052 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T10:16:07.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T10:16:07.053 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T10:16:07.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T10:16:07.062 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T10:16:34.781 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6871136032779103) > weight (0.25) 2021-02-20T10:16:34.789 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5817507012454745) > weight (0.25) 2021-02-20T10:16:34.789 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5447083422218879) > weight (0.25) 2021-02-20T10:16:34.790 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5716862428470161) > weight (0.25) 2021-02-20T10:16:34.790 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T10:17:14.538 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T10:17:14.534+0000 7fb533c45700 -1 reset not still connected to 0x558b8d9fdee0 2021-02-20T10:17:14.539 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T10:17:14.534+0000 7fb533c45700 -1 reset not still connected to 0x558b8d9fe080 2021-02-20T10:17:14.539 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T10:17:14.534+0000 7fb533c45700 -1 reset not still connected to 0x558b94650750 2021-02-20T10:17:14.539 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T10:17:14.534+0000 7fb533c45700 -1 reset not still connected to 0x558b94651d40 2021-02-20T10:17:14.539 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T10:17:14.534+0000 7fb533c45700 -1 reset not still connected to 0x558b94652970 2021-02-20T10:17:14.540 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T10:17:14.534+0000 7fb533c45700 -1 reset not still connected to 0x558b94b91260 2021-02-20T10:17:14.540 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T10:17:14.534+0000 7fb533c45700 -1 reset not still connected to 0x558b9a127110 2021-02-20T10:17:14.540 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T10:17:14.534+0000 7fb533c45700 -1 reset not still connected to 0x558b9a127ba0 2021-02-20T10:17:28.015 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T10:17:28.014+0000 7fb533c45700 -1 reset not still connected to 0x558b95d3fa80 2021-02-20T10:18:00.064 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:18:00.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:18:00.141 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T10:18:00.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T10:18:00.138+0000 7f0430c83700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 39291) UID: 1000 2021-02-20T10:18:00.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:18:06.141 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:18:06.147 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:18:06.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:18:06.148 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:18:06.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:18:06.148 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:18:06.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:18:06.151 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T10:18:11.790 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6047576888111192) > weight (0.25) 2021-02-20T10:18:11.791 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3985944629464513) > weight (0.25) 2021-02-20T10:18:11.791 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9089040400874633) > weight (0.25) 2021-02-20T10:18:11.791 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4166891805853271) > weight (0.25) 2021-02-20T10:18:11.792 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T10:19:21.152 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3864040650481867) > weight (0.25) 2021-02-20T10:19:21.162 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T10:19:21.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T10:19:21.163 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T10:19:21.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T10:19:21.161+0000 7f741e683700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 38190) UID: 1000 2021-02-20T10:19:21.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:19:23.793 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.494871207735064) > weight (0.25) 2021-02-20T10:19:23.799 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5631188791410153) > weight (0.25) 2021-02-20T10:19:23.800 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T10:19:23.800 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T10:19:23.801 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T10:19:23.801 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T10:19:23.801+0000 7f8d84f67700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 38924) UID: 1000 2021-02-20T10:19:24.002 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:19:24.163 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T10:19:24.164 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:19:24.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:19:24.165 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T10:19:24.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T10:19:24.166 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T10:19:24.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T10:19:24.170 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T10:19:30.802 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T10:19:30.809 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:19:30.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:19:30.809 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T10:19:30.810 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T10:19:30.811 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T10:19:30.814 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T10:19:30.814 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T10:21:09.815 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.49324816525111226) > weight (0.25) 2021-02-20T10:21:09.848 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8758482780630874) > weight (0.25) 2021-02-20T10:21:09.848 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5999531158349954) > weight (0.25) 2021-02-20T10:21:09.849 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7790728625456869) > weight (0.25) 2021-02-20T10:21:09.849 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T10:21:20.175 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3015220046491398) > weight (0.25) 2021-02-20T10:21:20.183 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.28619453451762045) > weight (0.25) 2021-02-20T10:21:20.183 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6546388286513709) > weight (0.25) 2021-02-20T10:21:20.184 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.317712779569541) > weight (0.25) 2021-02-20T10:21:20.184 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T10:22:16.850 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4203051073690074) > weight (0.25) 2021-02-20T10:22:16.858 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6347203241551503) > weight (0.25) 2021-02-20T10:22:16.858 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.623793297865663) > weight (0.25) 2021-02-20T10:22:16.858 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T10:22:16.859 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T10:22:16.860 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T10:22:16.861 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T10:22:16.858+0000 7f49b7f3a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 39572) UID: 1000 2021-02-20T10:22:17.261 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:22:24.861 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T10:22:24.868 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:22:24.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:22:24.870 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T10:22:24.870 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T10:22:24.872 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T10:22:24.875 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T10:22:24.876 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T10:22:35.185 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6913472003614815) > weight (0.25) 2021-02-20T10:22:35.193 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8365745915757613) > weight (0.25) 2021-02-20T10:22:35.194 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4709101518665124) > weight (0.25) 2021-02-20T10:22:35.194 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9350678018273607) > weight (0.25) 2021-02-20T10:22:35.194 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T10:23:31.877 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8187858277357677) > weight (0.25) 2021-02-20T10:23:31.916 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6522766939986626) > weight (0.25) 2021-02-20T10:23:31.916 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5954356965329929) > weight (0.25) 2021-02-20T10:23:31.916 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6197098400196628) > weight (0.25) 2021-02-20T10:23:31.917 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T10:23:47.195 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5427655383210505) > weight (0.25) 2021-02-20T10:23:47.202 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6701341763267759) > weight (0.25) 2021-02-20T10:23:47.202 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T10:23:47.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T10:23:47.204 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T10:23:47.204 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T10:23:47.204 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:23:47.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:23:47.204+0000 7f213b60a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 39757) UID: 1000 2021-02-20T10:23:47.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:23:53.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:23:53.328 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T10:23:53.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T10:23:53.330 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T10:23:53.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T10:23:53.334 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T10:24:54.335 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:24:54.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:24:54.345 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T10:24:54.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T10:24:54.343+0000 7f03b6e2d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 39851) UID: 1000 2021-02-20T10:24:54.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:24:55.346 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:24:55.347 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:24:55.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:24:55.347 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:24:55.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:24:55.348 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:24:55.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:24:55.362 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T10:25:18.918 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5513772902183066) > weight (0.25) 2021-02-20T10:25:18.929 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5720402214086464) > weight (0.25) 2021-02-20T10:25:18.929 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5193675756433073) > weight (0.25) 2021-02-20T10:25:18.929 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6977549967003795) > weight (0.25) 2021-02-20T10:25:18.930 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T10:26:25.931 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7398515452693758) > weight (0.25) 2021-02-20T10:26:25.972 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4504483868978999) > weight (0.25) 2021-02-20T10:26:25.973 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9306065076923041) > weight (0.25) 2021-02-20T10:26:25.973 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6313230577575434) > weight (0.25) 2021-02-20T10:26:25.973 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T10:26:51.363 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.45181194664653135) > weight (0.25) 2021-02-20T10:26:51.372 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9397308306488851) > weight (0.25) 2021-02-20T10:26:51.372 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9328531669913512) > weight (0.25) 2021-02-20T10:26:51.373 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7519179279315122) > weight (0.25) 2021-02-20T10:26:51.373 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T10:28:10.974 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:28:10.976 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:28:10.976 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T10:28:10.977 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T10:28:10.976+0000 7fedf7e91700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 39382) UID: 1000 2021-02-20T10:28:11.377 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:28:17.977 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:28:17.995 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:28:17.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:28:17.996 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:28:17.997 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:28:17.997 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:28:18.002 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:28:18.003 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T10:28:39.374 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5089728399557877) > weight (0.25) 2021-02-20T10:28:39.382 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T10:28:39.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T10:28:39.383 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T10:28:39.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T10:28:39.383+0000 7f426a6db700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 39942) UID: 1000 2021-02-20T10:28:39.529 INFO:tasks.ceph.cluster2.osd.0.smithi090.stderr:2021-02-20T10:28:39.527+0000 7f7bd2757700 -1 reset not still connected to 0x5624ba76ed80 2021-02-20T10:28:39.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:28:40.384 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T10:28:40.385 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:28:40.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:28:40.386 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T10:28:40.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T10:28:40.387 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T10:28:40.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T10:28:40.399 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T10:29:36.004 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:29:36.012 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:29:36.012 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T10:29:36.012 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:29:36.013 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:29:36.014 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T10:29:36.010+0000 7f800aec1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 40418) UID: 1000 2021-02-20T10:29:36.213 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:29:42.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:29:42.123 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:29:42.123 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:29:42.124 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:29:42.127 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:29:42.128 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T10:29:54.401 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2910714241456831) > weight (0.25) 2021-02-20T10:29:54.401 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T10:29:54.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T10:29:54.402 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T10:29:54.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T10:29:54.402+0000 7f94191f6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 40509) UID: 1000 2021-02-20T10:29:54.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:30:02.404 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T10:30:02.413 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:30:02.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:30:02.414 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T10:30:02.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T10:30:02.415 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T10:30:02.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T10:30:02.419 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T10:30:54.129 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.36408320646714354) > weight (0.25) 2021-02-20T10:30:54.130 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T10:30:54.131 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T10:30:54.131 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T10:30:54.139 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T10:30:54.133+0000 7fd2245fd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 39663) UID: 1000 2021-02-20T10:30:54.537 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:30:59.132 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T10:30:59.133 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:30:59.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:30:59.133 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T10:30:59.134 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T10:30:59.134 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T10:30:59.137 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T10:30:59.137 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T10:31:24.420 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:31:24.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:31:24.431 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T10:31:24.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T10:31:24.424+0000 7fa5229d7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 40316) UID: 1000 2021-02-20T10:31:24.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:31:32.431 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:31:32.432 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:31:32.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:31:32.433 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:31:32.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:31:32.433 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:31:32.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:31:32.439 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T10:32:03.139 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8342124307605248) > weight (0.25) 2021-02-20T10:32:03.147 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.29168507243070463) > weight (0.25) 2021-02-20T10:32:03.148 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T10:32:03.149 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T10:32:03.149 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T10:32:03.151 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T10:32:03.148+0000 7f39a4433700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 40041) UID: 1000 2021-02-20T10:32:03.351 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:32:06.151 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T10:32:06.159 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:32:06.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:32:06.160 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T10:32:06.161 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T10:32:06.161 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T10:32:06.166 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T10:32:06.166 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T10:33:06.168 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2795273955827313) > weight (0.25) 2021-02-20T10:33:06.177 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6942057990319441) > weight (0.25) 2021-02-20T10:33:06.178 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.39619725469777867) > weight (0.25) 2021-02-20T10:33:06.178 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7760499538959282) > weight (0.25) 2021-02-20T10:33:06.179 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T10:33:07.441 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5360253705363377) > weight (0.25) 2021-02-20T10:33:07.441 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6416458860930099) > weight (0.25) 2021-02-20T10:33:07.441 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8235082417533357) > weight (0.25) 2021-02-20T10:33:07.441 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T10:33:07.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T10:33:07.443 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T10:33:07.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T10:33:07.439+0000 7fe390bc8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 39481) UID: 1000 2021-02-20T10:33:07.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:33:14.444 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T10:33:14.451 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:33:14.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:33:14.452 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T10:33:14.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T10:33:14.452 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T10:33:14.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T10:33:14.455 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T10:34:18.457 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.32191893249170234) > weight (0.25) 2021-02-20T10:34:18.466 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T10:34:18.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T10:34:18.468 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T10:34:18.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T10:34:18.461+0000 7ff29a598700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 40700) UID: 1000 2021-02-20T10:34:18.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:34:22.469 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T10:34:22.470 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:34:22.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:34:22.470 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T10:34:22.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T10:34:22.471 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T10:34:22.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T10:34:22.479 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T10:34:22.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.757+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75366f00 [3/00a349fd-b244-4997-98f5-d4af9b72b8bc] start: already running 2021-02-20T10:34:22.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.757+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75367680 [3/115af1ac-f2c4-457c-acd0-ffd858708db6] start: already running 2021-02-20T10:34:22.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.757+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75367900 [3/3cee2b48-5f6f-4010-a2b4-5769e61ce00b] start: already running 2021-02-20T10:34:22.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.757+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75367b80 [3/3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd] start: already running 2021-02-20T10:34:22.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.757+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75367e00 [3/4821c43f-8dc9-4295-9d32-f4908f5024a2] start: already running 2021-02-20T10:34:22.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.757+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75368080 [3/498ebb02-e981-4c70-9666-80ebcdecb22e] start: already running 2021-02-20T10:34:22.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.757+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75368300 [3/4bc5386d-f47f-4712-9ca8-75fe41f4e72e] start: already running 2021-02-20T10:34:22.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.757+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75368580 [3/5cc4eeb6-e809-410f-8db4-a0f6f871e5be] start: already running 2021-02-20T10:34:22.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.757+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75368800 [3/621fdda4-5f94-4a13-8d07-ff942765191e] start: already running 2021-02-20T10:34:22.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.757+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75368a80 [3/7391691a-8b26-4d17-9cb3-e70e9b15d323] start: already running 2021-02-20T10:34:22.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.757+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75368d00 [3/8d0e16c9-d3d2-4b9c-a2e9-c0ac1f36de1a] start: already running 2021-02-20T10:34:22.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.757+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75368f80 [3/8ee65cf4-12ca-4ca6-b1e7-745c00784f3d] start: already running 2021-02-20T10:34:22.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.757+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75369200 [3/9781de1c-0ffd-4a24-ae0e-180268ec6066] start: already running 2021-02-20T10:34:22.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.757+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75369480 [3/a39316e2-bbbe-40ae-9620-51f282c47f6c] start: already running 2021-02-20T10:34:22.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.757+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e75368f80 [3/8ee65cf4-12ca-4ca6-b1e7-745c00784f3d] start: already running 2021-02-20T10:34:22.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.757+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e75369200 [3/9781de1c-0ffd-4a24-ae0e-180268ec6066] start: already running 2021-02-20T10:34:22.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.757+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e75369480 [3/a39316e2-bbbe-40ae-9620-51f282c47f6c] start: already running 2021-02-20T10:34:22.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.757+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e75369700 [3/a4f4a9dd-701f-4734-a679-4beea65adecf] start: already running 2021-02-20T10:34:22.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.757+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e75369980 [3/b18825d4-ad79-43d5-95f8-bbc84838ad3c] start: already running 2021-02-20T10:34:22.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.757+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e75369c00 [3/b614e8f1-3d22-46ce-9b70-32b4f254320a] start: already running 2021-02-20T10:34:22.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.757+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e793b2000 [3/c64947f6-af7d-4dad-b5bd-4c3ed0c6fde9] start: already running 2021-02-20T10:34:22.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.785+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75369700 [3/a4f4a9dd-701f-4734-a679-4beea65adecf] start: already running 2021-02-20T10:34:22.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.785+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75366f00 [3/00a349fd-b244-4997-98f5-d4af9b72b8bc] start: already running 2021-02-20T10:34:22.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.785+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75367680 [3/115af1ac-f2c4-457c-acd0-ffd858708db6] start: already running 2021-02-20T10:34:22.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.785+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75367900 [3/3cee2b48-5f6f-4010-a2b4-5769e61ce00b] start: already running 2021-02-20T10:34:22.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.785+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75367b80 [3/3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd] start: already running 2021-02-20T10:34:22.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.785+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75367e00 [3/4821c43f-8dc9-4295-9d32-f4908f5024a2] start: already running 2021-02-20T10:34:22.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.785+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75368080 [3/498ebb02-e981-4c70-9666-80ebcdecb22e] start: already running 2021-02-20T10:34:22.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.785+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75368300 [3/4bc5386d-f47f-4712-9ca8-75fe41f4e72e] start: already running 2021-02-20T10:34:22.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.785+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75368580 [3/5cc4eeb6-e809-410f-8db4-a0f6f871e5be] start: already running 2021-02-20T10:34:22.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.785+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75368800 [3/621fdda4-5f94-4a13-8d07-ff942765191e] start: already running 2021-02-20T10:34:22.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.785+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75368a80 [3/7391691a-8b26-4d17-9cb3-e70e9b15d323] start: already running 2021-02-20T10:34:22.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.785+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75368d00 [3/8d0e16c9-d3d2-4b9c-a2e9-c0ac1f36de1a] start: already running 2021-02-20T10:34:22.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.785+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e75368080 [3/498ebb02-e981-4c70-9666-80ebcdecb22e] start: already running 2021-02-20T10:34:22.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.785+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75368300 [3/4bc5386d-f47f-4712-9ca8-75fe41f4e72e] start: already running 2021-02-20T10:34:22.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.785+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e75368580 [3/5cc4eeb6-e809-410f-8db4-a0f6f871e5be] start: already running 2021-02-20T10:34:22.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.785+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e75368800 [3/621fdda4-5f94-4a13-8d07-ff942765191e] start: already running 2021-02-20T10:34:22.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.785+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e75368a80 [3/7391691a-8b26-4d17-9cb3-e70e9b15d323] start: already running 2021-02-20T10:34:22.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.789+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75368d00 [3/8d0e16c9-d3d2-4b9c-a2e9-c0ac1f36de1a] start: already running 2021-02-20T10:34:22.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.789+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75368f80 [3/8ee65cf4-12ca-4ca6-b1e7-745c00784f3d] start: already running 2021-02-20T10:34:22.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.789+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75369200 [3/9781de1c-0ffd-4a24-ae0e-180268ec6066] start: already running 2021-02-20T10:34:22.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.789+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75369480 [3/a39316e2-bbbe-40ae-9620-51f282c47f6c] start: already running 2021-02-20T10:34:22.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.789+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75369700 [3/a4f4a9dd-701f-4734-a679-4beea65adecf] start: already running 2021-02-20T10:34:22.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.789+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75369980 [3/b18825d4-ad79-43d5-95f8-bbc84838ad3c] start: already running 2021-02-20T10:34:22.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.789+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75369c00 [3/b614e8f1-3d22-46ce-9b70-32b4f254320a] start: already running 2021-02-20T10:34:22.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.789+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e793b2000 [3/c64947f6-af7d-4dad-b5bd-4c3ed0c6fde9] start: already running 2021-02-20T10:34:22.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.789+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e793b3180 [3/c82eb13a-1d88-4f85-a9e5-d88cf352a989] start: already running 2021-02-20T10:34:22.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.789+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e793b3400 [3/cf866930-b14f-41b9-b799-14620989c40a] start: already running 2021-02-20T10:34:22.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.789+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e793b3680 [3/e3d847a5-aed4-4f94-8c28-1456dc5219d3] start: already running 2021-02-20T10:34:22.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.789+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e793b3900 [3/eac102d0-8916-41aa-b29e-e80570397cac] start: already running 2021-02-20T10:34:22.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.789+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e793b3b80 [3/ecd57dfe-8ff1-46ab-842a-360eca32b968] start: already running 2021-02-20T10:34:22.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.789+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e793b3e00 [3/fbe49d75-97ae-46ab-9f18-5816b4adfbe7] start: already running 2021-02-20T10:34:22.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.789+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e793b4080 [3/fde20882-9f89-4465-9521-7843f57ae5d4] start: already running 2021-02-20T10:34:22.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.801+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e793b3180 [3/c82eb13a-1d88-4f85-a9e5-d88cf352a989] start: already running 2021-02-20T10:34:22.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.801+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e793b3400 [3/cf866930-b14f-41b9-b799-14620989c40a] start: already running 2021-02-20T10:34:22.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.801+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e793b3680 [3/e3d847a5-aed4-4f94-8c28-1456dc5219d3] start: already running 2021-02-20T10:34:22.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.801+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e793b3900 [3/eac102d0-8916-41aa-b29e-e80570397cac] start: already running 2021-02-20T10:34:22.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.801+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e793b3b80 [3/ecd57dfe-8ff1-46ab-842a-360eca32b968] start: already running 2021-02-20T10:34:22.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.801+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e793b3e00 [3/fbe49d75-97ae-46ab-9f18-5816b4adfbe7] start: already running 2021-02-20T10:34:22.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.801+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75366f00 [3/00a349fd-b244-4997-98f5-d4af9b72b8bc] start: already running 2021-02-20T10:34:22.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.801+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e75367680 [3/115af1ac-f2c4-457c-acd0-ffd858708db6] start: already running 2021-02-20T10:34:22.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.801+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75367900 [3/3cee2b48-5f6f-4010-a2b4-5769e61ce00b] start: already running 2021-02-20T10:34:22.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.801+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75367e00 [3/4821c43f-8dc9-4295-9d32-f4908f5024a2] start: already running 2021-02-20T10:34:22.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.801+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75368d00 [3/8d0e16c9-d3d2-4b9c-a2e9-c0ac1f36de1a] start: already running 2021-02-20T10:34:22.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.801+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75368f80 [3/8ee65cf4-12ca-4ca6-b1e7-745c00784f3d] start: already running 2021-02-20T10:34:22.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.801+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e75369200 [3/9781de1c-0ffd-4a24-ae0e-180268ec6066] start: already running 2021-02-20T10:34:22.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.801+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e75369480 [3/a39316e2-bbbe-40ae-9620-51f282c47f6c] start: already running 2021-02-20T10:34:22.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.801+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e75369700 [3/a4f4a9dd-701f-4734-a679-4beea65adecf] start: already running 2021-02-20T10:34:22.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.801+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e75369980 [3/b18825d4-ad79-43d5-95f8-bbc84838ad3c] start: already running 2021-02-20T10:34:22.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.813+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e75369c00 [3/b614e8f1-3d22-46ce-9b70-32b4f254320a] start: already running 2021-02-20T10:34:22.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.813+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e793b2000 [3/c64947f6-af7d-4dad-b5bd-4c3ed0c6fde9] start: already running 2021-02-20T10:34:22.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.813+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e793b3180 [3/c82eb13a-1d88-4f85-a9e5-d88cf352a989] start: already running 2021-02-20T10:34:22.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.813+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e793b3400 [3/cf866930-b14f-41b9-b799-14620989c40a] start: already running 2021-02-20T10:34:22.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.813+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e793b3680 [3/e3d847a5-aed4-4f94-8c28-1456dc5219d3] start: already running 2021-02-20T10:34:22.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.813+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e793b3900 [3/eac102d0-8916-41aa-b29e-e80570397cac] start: already running 2021-02-20T10:34:22.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.813+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e793b3b80 [3/ecd57dfe-8ff1-46ab-842a-360eca32b968] start: already running 2021-02-20T10:34:22.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.813+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e793b3e00 [3/fbe49d75-97ae-46ab-9f18-5816b4adfbe7] start: already running 2021-02-20T10:34:22.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.813+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e793b4080 [3/fde20882-9f89-4465-9521-7843f57ae5d4] start: already running 2021-02-20T10:34:22.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.813+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75368d00 [3/8d0e16c9-d3d2-4b9c-a2e9-c0ac1f36de1a] start: already running 2021-02-20T10:34:22.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.813+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e75368f80 [3/8ee65cf4-12ca-4ca6-b1e7-745c00784f3d] start: already running 2021-02-20T10:34:22.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.813+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e75369200 [3/9781de1c-0ffd-4a24-ae0e-180268ec6066] start: already running 2021-02-20T10:34:22.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.813+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e75369480 [3/a39316e2-bbbe-40ae-9620-51f282c47f6c] start: already running 2021-02-20T10:34:22.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.813+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75369700 [3/a4f4a9dd-701f-4734-a679-4beea65adecf] start: already running 2021-02-20T10:34:22.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.813+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75369980 [3/b18825d4-ad79-43d5-95f8-bbc84838ad3c] start: already running 2021-02-20T10:34:22.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.813+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e75369c00 [3/b614e8f1-3d22-46ce-9b70-32b4f254320a] start: already running 2021-02-20T10:34:22.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.813+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e793b2000 [3/c64947f6-af7d-4dad-b5bd-4c3ed0c6fde9] start: already running 2021-02-20T10:34:22.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.813+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e793b3180 [3/c82eb13a-1d88-4f85-a9e5-d88cf352a989] start: already running 2021-02-20T10:34:22.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.817+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e793b3400 [3/cf866930-b14f-41b9-b799-14620989c40a] start: already running 2021-02-20T10:34:22.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.817+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e793b3680 [3/e3d847a5-aed4-4f94-8c28-1456dc5219d3] start: already running 2021-02-20T10:34:22.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.817+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e793b3900 [3/eac102d0-8916-41aa-b29e-e80570397cac] start: already running 2021-02-20T10:34:22.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.817+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e793b3b80 [3/ecd57dfe-8ff1-46ab-842a-360eca32b968] start: already running 2021-02-20T10:34:22.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.817+0000 7f6e4fc0b700 -1 rbd::mirror::ImageReplayer: 0x561e793b3e00 [3/fbe49d75-97ae-46ab-9f18-5816b4adfbe7] start: already running 2021-02-20T10:34:22.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:34:22.817+0000 7f6e5040c700 -1 rbd::mirror::ImageReplayer: 0x561e793b4080 [3/fde20882-9f89-4465-9521-7843f57ae5d4] start: already running 2021-02-20T10:34:26.180 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3294790208997844) > weight (0.25) 2021-02-20T10:34:26.180 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.27993471501546385) > weight (0.25) 2021-02-20T10:34:26.181 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.773981616531548) > weight (0.25) 2021-02-20T10:34:26.181 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T10:34:26.182 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T10:34:26.182 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T10:34:26.184 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T10:34:26.177+0000 7f5f0e41f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 40133) UID: 1000 2021-02-20T10:34:26.585 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:34:31.183 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T10:34:31.192 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:34:31.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:34:31.193 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T10:34:31.193 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T10:34:31.194 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T10:34:31.197 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T10:34:31.197 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T10:35:55.198 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9411206580778818) > weight (0.25) 2021-02-20T10:35:55.205 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T10:35:55.206 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T10:35:55.207 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T10:35:55.208 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T10:35:55.204+0000 7f5568631700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 40791) UID: 1000 2021-02-20T10:35:55.408 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:36:00.208 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T10:36:00.208 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:36:00.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:36:00.209 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T10:36:00.210 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T10:36:00.210 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T10:36:00.214 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T10:36:00.214 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T10:36:07.480 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:36:07.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:36:07.489 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T10:36:07.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T10:36:07.488+0000 7f8c6d625700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 40882) UID: 1000 2021-02-20T10:36:07.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:36:12.491 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:36:12.498 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:36:12.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:36:12.498 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:36:12.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:36:12.500 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:36:12.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:36:12.503 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T10:37:46.215 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5976521018120251) > weight (0.25) 2021-02-20T10:37:46.226 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7471123464702778) > weight (0.25) 2021-02-20T10:37:46.227 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T10:37:46.228 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T10:37:46.229 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T10:37:46.230 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T10:37:46.226+0000 7f183e9b8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 40973) UID: 1000 2021-02-20T10:37:46.430 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:37:49.229 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T10:37:49.235 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:37:49.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:37:49.236 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T10:37:49.236 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T10:37:49.236 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T10:37:49.239 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T10:37:49.240 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T10:37:54.504 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:37:54.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:37:54.512 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T10:37:54.512 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:37:54.513 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:37:54.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T10:37:54.506+0000 7fa74ac06700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 41449) UID: 1000 2021-02-20T10:37:54.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:38:00.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:38:00.623 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:38:00.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:38:00.624 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:38:00.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:38:00.627 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T10:39:07.628 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4908137610300253) > weight (0.25) 2021-02-20T10:39:07.636 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3499160670240825) > weight (0.25) 2021-02-20T10:39:07.636 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4583076022933116) > weight (0.25) 2021-02-20T10:39:07.637 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9016322752770751) > weight (0.25) 2021-02-20T10:39:07.637 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T10:39:11.241 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7170079594375142) > weight (0.25) 2021-02-20T10:39:11.242 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T10:39:11.243 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T10:39:11.243 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T10:39:11.244 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T10:39:11.244 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:39:11.248 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T10:39:11.244+0000 7f4a99011700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 41358) UID: 1000 2021-02-20T10:39:11.453 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:39:17.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:39:17.346 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T10:39:17.346 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T10:39:17.346 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T10:39:17.349 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T10:39:17.349 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T10:40:28.351 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8863529926532334) > weight (0.25) 2021-02-20T10:40:28.360 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.671322990637591) > weight (0.25) 2021-02-20T10:40:28.360 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T10:40:28.361 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T10:40:28.362 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T10:40:28.362 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T10:40:28.363 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:40:28.363 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T10:40:28.359+0000 7f50ec7ea700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 41541) UID: 1000 2021-02-20T10:40:28.563 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:40:34.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:40:34.471 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T10:40:34.472 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T10:40:34.472 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T10:40:34.475 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T10:40:34.475 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T10:40:42.389 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T10:40:42.384+0000 7fb533c45700 -1 reset not still connected to 0x558b91bf5380 2021-02-20T10:40:42.389 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T10:40:42.384+0000 7fb533c45700 -1 reset not still connected to 0x558b9a129a80 2021-02-20T10:40:56.638 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.664733749548497) > weight (0.25) 2021-02-20T10:40:56.715 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4066858495798391) > weight (0.25) 2021-02-20T10:40:56.715 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9143357209686758) > weight (0.25) 2021-02-20T10:40:56.716 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7783462792155474) > weight (0.25) 2021-02-20T10:40:56.716 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T10:41:35.477 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7387229419386863) > weight (0.25) 2021-02-20T10:41:35.489 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T10:41:35.490 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T10:41:35.490 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T10:41:35.492 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T10:41:35.486+0000 7f2976124700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 41723) UID: 1000 2021-02-20T10:41:35.692 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:41:44.492 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T10:41:44.498 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:41:44.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:41:44.499 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T10:41:44.500 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T10:41:44.501 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T10:41:44.504 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T10:41:44.504 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T10:42:05.716 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:42:05.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:42:05.735 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T10:42:05.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T10:42:05.721+0000 7fb66b9f9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 41632) UID: 1000 2021-02-20T10:42:05.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:42:11.736 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:42:11.744 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:42:11.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:42:11.745 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:42:11.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:42:11.746 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:42:11.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:42:11.749 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T10:42:56.505 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4241291369417586) > weight (0.25) 2021-02-20T10:42:56.512 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6523685944958786) > weight (0.25) 2021-02-20T10:42:56.513 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T10:42:56.514 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T10:42:56.514 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T10:42:56.515 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T10:42:56.515 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:42:56.516 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T10:42:56.509+0000 7f97fc6ce700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 41815) UID: 1000 2021-02-20T10:42:56.715 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:43:02.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:43:02.624 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T10:43:02.624 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T10:43:02.625 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T10:43:02.629 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T10:43:02.629 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T10:43:25.750 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.29532134967888846) > weight (0.25) 2021-02-20T10:43:25.758 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T10:43:25.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T10:43:25.760 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T10:43:25.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T10:43:25.756+0000 7f8bc9d0c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 41163) UID: 1000 2021-02-20T10:43:25.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:43:33.761 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T10:43:33.765 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:43:33.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:43:33.766 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T10:43:33.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T10:43:33.766 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T10:43:33.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T10:43:33.770 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T10:44:56.630 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7287166365284738) > weight (0.25) 2021-02-20T10:44:56.638 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8782276022578895) > weight (0.25) 2021-02-20T10:44:56.638 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7250769757554283) > weight (0.25) 2021-02-20T10:44:56.639 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8536285139658307) > weight (0.25) 2021-02-20T10:44:56.639 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T10:45:29.773 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4111672085445197) > weight (0.25) 2021-02-20T10:45:29.773 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9859460485773703) > weight (0.25) 2021-02-20T10:45:29.774 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3964982029702089) > weight (0.25) 2021-02-20T10:45:29.774 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6418686005492489) > weight (0.25) 2021-02-20T10:45:29.774 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T10:46:21.641 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:46:21.648 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:46:21.648 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T10:46:21.652 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T10:46:21.641+0000 7f204033d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 40608) UID: 1000 2021-02-20T10:46:22.050 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:46:25.649 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:46:25.650 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:46:25.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:46:25.651 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:46:25.651 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:46:25.652 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:46:25.654 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:46:25.655 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T10:47:27.775 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:47:27.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:47:27.872 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T10:47:27.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T10:47:27.868+0000 7efcbc2f8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 41997) UID: 1000 2021-02-20T10:47:28.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:47:32.874 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:47:32.879 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:47:32.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:47:32.880 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:47:32.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:47:32.880 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:47:32.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:47:32.884 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T10:47:36.656 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8518185414941738) > weight (0.25) 2021-02-20T10:47:36.657 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T10:47:36.657 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T10:47:36.658 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T10:47:36.658 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T10:47:36.656+0000 7fda0ae05700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 41906) UID: 1000 2021-02-20T10:47:36.859 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:47:40.659 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T10:47:40.659 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:47:40.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:47:40.660 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T10:47:40.660 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T10:47:40.661 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T10:47:40.663 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T10:47:40.664 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T10:49:20.884 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6081999358680631) > weight (0.25) 2021-02-20T10:49:20.885 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7100591769030133) > weight (0.25) 2021-02-20T10:49:20.886 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T10:49:20.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T10:49:20.887 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T10:49:20.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:49:20.882+0000 7f6e50c0d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 40224) UID: 1000 2021-02-20T10:49:21.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:49:24.665 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4925544606286789) > weight (0.25) 2021-02-20T10:49:24.665 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8843240425351783) > weight (0.25) 2021-02-20T10:49:24.666 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9939824556398218) > weight (0.25) 2021-02-20T10:49:24.666 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T10:49:24.666 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T10:49:24.667 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T10:49:24.668 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T10:49:24.662+0000 7f9935272700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 41262) UID: 1000 2021-02-20T10:49:24.868 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:49:29.888 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T10:49:29.888 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:49:29.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:49:29.889 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T10:49:29.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T10:49:29.890 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T10:49:29.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T10:49:29.893 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T10:49:30.667 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T10:49:30.668 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:49:30.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:49:30.669 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T10:49:30.669 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T10:49:30.669 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T10:49:30.671 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T10:49:30.672 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T10:50:32.894 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:50:32.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:50:32.903 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T10:50:32.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T10:50:32.897+0000 7f386ec0f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 42365) UID: 1000 2021-02-20T10:50:33.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:50:33.904 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:50:33.905 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:50:33.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:50:33.905 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:50:33.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:50:33.906 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:50:33.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:50:33.909 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T10:50:51.673 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7024643311329543) > weight (0.25) 2021-02-20T10:50:51.680 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.28636759474670437) > weight (0.25) 2021-02-20T10:50:51.680 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.40724583407149373) > weight (0.25) 2021-02-20T10:50:51.681 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T10:50:51.681 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T10:50:51.682 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T10:50:51.682 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T10:50:51.680+0000 7f80a57ac700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 42647) UID: 1000 2021-02-20T10:50:51.883 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:50:56.682 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T10:50:56.683 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:50:56.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:50:56.684 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T10:50:56.684 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T10:50:56.684 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T10:50:56.687 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T10:50:56.688 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T10:52:25.910 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8865837677896188) > weight (0.25) 2021-02-20T10:52:25.911 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T10:52:25.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T10:52:25.912 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T10:52:25.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T10:52:25.911+0000 7f2d872e9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 42179) UID: 1000 2021-02-20T10:52:26.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:52:32.914 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T10:52:32.914 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:52:32.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:52:32.915 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T10:52:32.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T10:52:32.916 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T10:52:32.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T10:52:32.919 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T10:52:49.688 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4204661301419951) > weight (0.25) 2021-02-20T10:52:49.689 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T10:52:49.690 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T10:52:49.690 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T10:52:49.691 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T10:52:49.686+0000 7fba9c91d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 42456) UID: 1000 2021-02-20T10:52:49.891 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:52:57.691 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T10:52:57.692 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:52:57.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:52:57.693 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T10:52:57.693 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T10:52:57.693 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T10:52:57.696 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T10:52:57.696 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T10:54:24.920 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6527997516266246) > weight (0.25) 2021-02-20T10:54:24.921 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T10:54:24.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T10:54:24.923 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T10:54:24.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T10:54:24.917+0000 7f3e7438e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 42929) UID: 1000 2021-02-20T10:54:25.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:54:31.924 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T10:54:31.925 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:54:31.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:54:31.927 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T10:54:31.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T10:54:31.928 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T10:54:31.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T10:54:31.935 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T10:54:52.697 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.29131723414225674) > weight (0.25) 2021-02-20T10:54:52.698 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7382305674692522) > weight (0.25) 2021-02-20T10:54:52.698 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T10:54:52.699 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T10:54:52.699 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T10:54:52.700 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T10:54:52.696+0000 7f7225e6e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 42088) UID: 1000 2021-02-20T10:54:53.101 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:54:54.701 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T10:54:54.701 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:54:54.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:54:54.702 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T10:54:54.702 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T10:54:54.703 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T10:54:54.705 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T10:54:54.706 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T10:56:09.707 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:56:09.708 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:56:09.709 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T10:56:09.710 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T10:56:09.707+0000 7f5769ff9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 42274) UID: 1000 2021-02-20T10:56:09.910 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:56:09.937 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8532434675990724) > weight (0.25) 2021-02-20T10:56:09.937 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.25309730091547966) > weight (0.25) 2021-02-20T10:56:09.937 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T10:56:09.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T10:56:09.938 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T10:56:09.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T10:56:09.963+0000 7f007c08a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 42556) UID: 1000 2021-02-20T10:56:10.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:56:13.710 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:56:13.710 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:56:13.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:56:13.711 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:56:13.711 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:56:13.712 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:56:13.715 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:56:13.715 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T10:56:17.939 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T10:56:17.940 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:56:17.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:56:17.940 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T10:56:17.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T10:56:17.941 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T10:56:17.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T10:56:17.944 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T10:58:04.716 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4840416981525365) > weight (0.25) 2021-02-20T10:58:04.725 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.44963054017175985) > weight (0.25) 2021-02-20T10:58:04.725 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4834093261125604) > weight (0.25) 2021-02-20T10:58:04.725 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7209298660940944) > weight (0.25) 2021-02-20T10:58:04.726 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T10:58:05.944 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:58:05.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:58:05.947 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T10:58:05.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T10:58:05.941+0000 7fdc4a6c2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 42739) UID: 1000 2021-02-20T10:58:06.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:58:06.948 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:58:06.949 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:58:06.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:58:06.950 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:58:06.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:58:06.951 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:58:06.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:58:06.954 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T10:59:07.726 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:59:07.746 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:59:07.746 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T10:59:07.747 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T10:59:07.744+0000 7fb8bcfd3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 43296) UID: 1000 2021-02-20T10:59:07.947 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:59:11.955 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9978161012023844) > weight (0.25) 2021-02-20T10:59:11.960 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T10:59:11.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T10:59:11.961 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T10:59:11.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T10:59:11.960+0000 7f8407c0a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 43111) UID: 1000 2021-02-20T10:59:12.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:59:14.747 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:59:14.747 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:59:14.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:59:14.748 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:59:14.748 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:59:14.749 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:59:14.752 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:59:14.752 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T10:59:19.962 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T10:59:19.966 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:59:19.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:59:19.967 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T10:59:19.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T10:59:19.968 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T10:59:19.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T10:59:19.971 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T11:00:56.753 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.696215634956954) > weight (0.25) 2021-02-20T11:00:56.754 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T11:00:56.755 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T11:00:56.755 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T11:00:56.757 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T11:00:56.750+0000 7fa63ceb8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 43020) UID: 1000 2021-02-20T11:00:57.159 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:01:00.756 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T11:01:00.756 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:01:00.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:01:00.757 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T11:01:00.757 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T11:01:00.758 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T11:01:00.762 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T11:01:00.762 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T11:01:03.971 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T11:01:03.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T11:01:03.973 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T11:01:03.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T11:01:03.970+0000 7f3802010700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 43479) UID: 1000 2021-02-20T11:01:04.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:01:05.974 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T11:01:05.974 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:01:05.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:01:05.975 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T11:01:05.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T11:01:05.976 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T11:01:05.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T11:01:05.979 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T11:02:30.763 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8694252182102284) > weight (0.25) 2021-02-20T11:02:30.764 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.33103606380106243) > weight (0.25) 2021-02-20T11:02:30.764 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5398151451243216) > weight (0.25) 2021-02-20T11:02:30.764 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T11:02:30.765 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T11:02:30.765 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T11:02:30.766 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T11:02:30.761+0000 7ff081b92700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 42830) UID: 1000 2021-02-20T11:02:30.967 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:02:35.766 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T11:02:35.767 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:02:35.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:02:35.768 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T11:02:35.768 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T11:02:35.768 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T11:02:35.771 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T11:02:35.771 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T11:02:43.980 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T11:02:43.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T11:02:43.981 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T11:02:43.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T11:02:43.972+0000 7f9a006b5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 43844) UID: 1000 2021-02-20T11:02:44.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:02:44.984 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T11:02:44.984 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:02:44.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:02:44.985 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T11:02:44.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T11:02:44.985 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T11:02:44.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T11:02:44.989 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T11:03:15.026 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T11:03:15.020+0000 7f7308f93700 -1 reset not still connected to 0x55aac2356820 2021-02-20T11:04:05.991 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T11:04:05.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T11:04:05.994 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T11:04:05.994 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T11:04:05.995 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:04:05.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T11:04:05.991+0000 7fd74786a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 44026) UID: 1000 2021-02-20T11:04:06.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:04:12.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:04:12.099 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T11:04:12.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T11:04:12.099 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T11:04:12.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T11:04:12.103 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T11:04:14.772 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T11:04:14.773 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T11:04:14.773 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T11:04:14.774 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T11:04:14.767+0000 7f11f545d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 43570) UID: 1000 2021-02-20T11:04:15.175 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:04:18.774 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T11:04:18.774 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:04:18.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:04:18.775 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T11:04:18.776 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T11:04:18.776 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T11:04:18.779 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T11:04:18.779 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T11:04:19.003 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:18.995+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x56486ce4f680 [3/3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd] start: already running 2021-02-20T11:04:19.018 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc0174a1700 -1 rbd::mirror::ImageReplayer: 0x56486ce50080 [3/5cc4eeb6-e809-410f-8db4-a0f6f871e5be] start: already running 2021-02-20T11:04:19.018 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc0174a1700 -1 rbd::mirror::ImageReplayer: 0x56486ce50300 [3/621fdda4-5f94-4a13-8d07-ff942765191e] start: already running 2021-02-20T11:04:19.018 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc0174a1700 -1 rbd::mirror::ImageReplayer: 0x56486ce50580 [3/7391691a-8b26-4d17-9cb3-e70e9b15d323] start: already running 2021-02-20T11:04:19.019 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc0174a1700 -1 rbd::mirror::ImageReplayer: 0x56486ce50800 [3/8d0e16c9-d3d2-4b9c-a2e9-c0ac1f36de1a] start: already running 2021-02-20T11:04:19.019 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc0174a1700 -1 rbd::mirror::ImageReplayer: 0x56486ce50a80 [3/8ee65cf4-12ca-4ca6-b1e7-745c00784f3d] start: already running 2021-02-20T11:04:19.019 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc0174a1700 -1 rbd::mirror::ImageReplayer: 0x56486ce50d00 [3/9781de1c-0ffd-4a24-ae0e-180268ec6066] start: already running 2021-02-20T11:04:19.020 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc0174a1700 -1 rbd::mirror::ImageReplayer: 0x56486ce50f80 [3/a39316e2-bbbe-40ae-9620-51f282c47f6c] start: already running 2021-02-20T11:04:19.020 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc0174a1700 -1 rbd::mirror::ImageReplayer: 0x56486ce51200 [3/a4f4a9dd-701f-4734-a679-4beea65adecf] start: already running 2021-02-20T11:04:19.020 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc0174a1700 -1 rbd::mirror::ImageReplayer: 0x56486ce51480 [3/b18825d4-ad79-43d5-95f8-bbc84838ad3c] start: already running 2021-02-20T11:04:19.020 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc0174a1700 -1 rbd::mirror::ImageReplayer: 0x56486ce51700 [3/b614e8f1-3d22-46ce-9b70-32b4f254320a] start: already running 2021-02-20T11:04:19.021 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc0174a1700 -1 rbd::mirror::ImageReplayer: 0x56486ce51980 [3/c64947f6-af7d-4dad-b5bd-4c3ed0c6fde9] start: already running 2021-02-20T11:04:19.021 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc0174a1700 -1 rbd::mirror::ImageReplayer: 0x56486ce51c00 [3/c82eb13a-1d88-4f85-a9e5-d88cf352a989] start: already running 2021-02-20T11:04:19.021 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc0174a1700 -1 rbd::mirror::ImageReplayer: 0x564870eba000 [3/cf866930-b14f-41b9-b799-14620989c40a] start: already running 2021-02-20T11:04:19.021 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc0174a1700 -1 rbd::mirror::ImageReplayer: 0x564870eba280 [3/e3d847a5-aed4-4f94-8c28-1456dc5219d3] start: already running 2021-02-20T11:04:19.021 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc0174a1700 -1 rbd::mirror::ImageReplayer: 0x564870eba500 [3/eac102d0-8916-41aa-b29e-e80570397cac] start: already running 2021-02-20T11:04:19.022 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc0174a1700 -1 rbd::mirror::ImageReplayer: 0x564870eba780 [3/ecd57dfe-8ff1-46ab-842a-360eca32b968] start: already running 2021-02-20T11:04:19.022 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc0174a1700 -1 rbd::mirror::ImageReplayer: 0x564870ebaa00 [3/fbe49d75-97ae-46ab-9f18-5816b4adfbe7] start: already running 2021-02-20T11:04:19.022 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc0174a1700 -1 rbd::mirror::ImageReplayer: 0x564870ebac80 [3/fde20882-9f89-4465-9521-7843f57ae5d4] start: already running 2021-02-20T11:04:19.022 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc0174a1700 -1 rbd::mirror::ImageReplayer: 0x56486ce4ef00 [3/00a349fd-b244-4997-98f5-d4af9b72b8bc] start: already running 2021-02-20T11:04:19.023 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x56486ce4f180 [3/115af1ac-f2c4-457c-acd0-ffd858708db6] start: already running 2021-02-20T11:04:19.023 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x56486ce4f400 [3/3cee2b48-5f6f-4010-a2b4-5769e61ce00b] start: already running 2021-02-20T11:04:19.024 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x56486ce4f900 [3/4821c43f-8dc9-4295-9d32-f4908f5024a2] start: already running 2021-02-20T11:04:19.024 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x56486ce4fb80 [3/498ebb02-e981-4c70-9666-80ebcdecb22e] start: already running 2021-02-20T11:04:19.024 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x56486ce4fe00 [3/4bc5386d-f47f-4712-9ca8-75fe41f4e72e] start: already running 2021-02-20T11:04:19.024 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x56486ce50080 [3/5cc4eeb6-e809-410f-8db4-a0f6f871e5be] start: already running 2021-02-20T11:04:19.025 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x56486ce50300 [3/621fdda4-5f94-4a13-8d07-ff942765191e] start: already running 2021-02-20T11:04:19.025 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x56486ce50580 [3/7391691a-8b26-4d17-9cb3-e70e9b15d323] start: already running 2021-02-20T11:04:19.025 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x56486ce50800 [3/8d0e16c9-d3d2-4b9c-a2e9-c0ac1f36de1a] start: already running 2021-02-20T11:04:19.025 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.011+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x56486ce50a80 [3/8ee65cf4-12ca-4ca6-b1e7-745c00784f3d] start: already running 2021-02-20T11:04:19.035 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.027+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x56486ce50d00 [3/9781de1c-0ffd-4a24-ae0e-180268ec6066] start: already running 2021-02-20T11:04:19.035 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.027+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x56486ce50f80 [3/a39316e2-bbbe-40ae-9620-51f282c47f6c] start: already running 2021-02-20T11:04:19.036 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.027+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x56486ce51200 [3/a4f4a9dd-701f-4734-a679-4beea65adecf] start: already running 2021-02-20T11:04:19.036 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.027+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x56486ce51480 [3/b18825d4-ad79-43d5-95f8-bbc84838ad3c] start: already running 2021-02-20T11:04:19.036 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.027+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x56486ce51700 [3/b614e8f1-3d22-46ce-9b70-32b4f254320a] start: already running 2021-02-20T11:04:19.036 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.027+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x56486ce51980 [3/c64947f6-af7d-4dad-b5bd-4c3ed0c6fde9] start: already running 2021-02-20T11:04:19.036 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.027+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x56486ce51c00 [3/c82eb13a-1d88-4f85-a9e5-d88cf352a989] start: already running 2021-02-20T11:04:19.037 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.027+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x564870eba000 [3/cf866930-b14f-41b9-b799-14620989c40a] start: already running 2021-02-20T11:04:19.037 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.027+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x564870eba280 [3/e3d847a5-aed4-4f94-8c28-1456dc5219d3] start: already running 2021-02-20T11:04:19.037 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.027+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x564870eba500 [3/eac102d0-8916-41aa-b29e-e80570397cac] start: already running 2021-02-20T11:04:19.037 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.027+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x564870eba780 [3/ecd57dfe-8ff1-46ab-842a-360eca32b968] start: already running 2021-02-20T11:04:19.037 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.027+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x564870ebaa00 [3/fbe49d75-97ae-46ab-9f18-5816b4adfbe7] start: already running 2021-02-20T11:04:19.038 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:04:19.027+0000 7fc017ca2700 -1 rbd::mirror::ImageReplayer: 0x564870ebac80 [3/fde20882-9f89-4465-9521-7843f57ae5d4] start: already running 2021-02-20T11:05:34.780 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.621715996595729) > weight (0.25) 2021-02-20T11:05:34.781 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.30102982525376776) > weight (0.25) 2021-02-20T11:05:34.781 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T11:05:34.782 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T11:05:34.782 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T11:05:34.784 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:05:34.778+0000 7fc0184a3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 43202) UID: 1000 2021-02-20T11:05:35.184 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:05:39.783 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T11:05:39.784 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:05:39.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:05:39.785 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T11:05:39.785 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T11:05:39.785 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T11:05:39.789 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T11:05:39.789 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T11:05:43.104 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.702650432715754) > weight (0.25) 2021-02-20T11:05:43.104 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T11:05:43.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T11:05:43.105 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T11:05:43.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T11:05:43.097+0000 7f4309cb7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 43661) UID: 1000 2021-02-20T11:05:43.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:05:48.106 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T11:05:48.107 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:05:48.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:05:48.107 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T11:05:48.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T11:05:48.108 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T11:05:48.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T11:05:48.111 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T11:07:19.790 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8015675995546513) > weight (0.25) 2021-02-20T11:07:19.791 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6407665043907013) > weight (0.25) 2021-02-20T11:07:19.792 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3924424908667107) > weight (0.25) 2021-02-20T11:07:19.792 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.622280836066577) > weight (0.25) 2021-02-20T11:07:19.792 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T11:07:32.111 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9902669591763934) > weight (0.25) 2021-02-20T11:07:32.112 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7501330734529965) > weight (0.25) 2021-02-20T11:07:32.112 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T11:07:32.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T11:07:32.114 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T11:07:32.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T11:07:32.108+0000 7fd7c8aae700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 43387) UID: 1000 2021-02-20T11:07:32.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:07:37.114 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T11:07:37.115 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:07:37.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:07:37.116 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T11:07:37.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T11:07:37.117 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T11:07:37.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T11:07:37.121 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T11:08:29.793 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.655057604224331) > weight (0.25) 2021-02-20T11:08:29.794 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.929264074729877) > weight (0.25) 2021-02-20T11:08:29.794 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.37256252349404684) > weight (0.25) 2021-02-20T11:08:29.795 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.40958379417232704) > weight (0.25) 2021-02-20T11:08:29.795 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T11:09:23.122 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T11:09:23.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T11:09:23.123 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T11:09:23.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T11:09:23.118+0000 7f9271931700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 44117) UID: 1000 2021-02-20T11:09:23.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:09:31.124 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T11:09:31.125 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:09:31.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:09:31.126 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T11:09:31.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T11:09:31.126 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T11:09:31.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T11:09:31.130 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T11:10:15.796 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.35662100202154423) > weight (0.25) 2021-02-20T11:10:15.797 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7388882292195271) > weight (0.25) 2021-02-20T11:10:15.798 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T11:10:15.798 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T11:10:15.799 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T11:10:15.799 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:10:15.793+0000 7f9736036700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 44305) UID: 1000 2021-02-20T11:10:16.000 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:10:23.800 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T11:10:23.801 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:10:23.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:10:23.802 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T11:10:23.803 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T11:10:23.803 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T11:10:23.806 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T11:10:23.807 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T11:10:54.131 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5813409094270232) > weight (0.25) 2021-02-20T11:10:54.131 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.31810243679433026) > weight (0.25) 2021-02-20T11:10:54.132 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.37791297441342264) > weight (0.25) 2021-02-20T11:10:54.132 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T11:10:54.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T11:10:54.133 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T11:10:54.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T11:10:54.128+0000 7ff978d4e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 41072) UID: 1000 2021-02-20T11:10:54.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:10:57.133 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T11:10:57.134 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:10:57.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:10:57.135 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T11:10:57.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T11:10:57.135 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T11:10:57.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T11:10:57.138 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T11:11:28.808 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7316427605453778) > weight (0.25) 2021-02-20T11:11:28.808 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6059352612954912) > weight (0.25) 2021-02-20T11:11:28.809 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T11:11:28.809 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T11:11:28.810 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T11:11:28.810 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T11:11:28.810 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:11:28.811 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:11:28.804+0000 7fd309010700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 44670) UID: 1000 2021-02-20T11:11:29.011 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:11:34.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:11:34.912 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T11:11:34.912 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T11:11:34.913 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T11:11:34.916 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T11:11:34.916 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T11:12:29.139 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6864477158101433) > weight (0.25) 2021-02-20T11:12:29.140 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7920488679346183) > weight (0.25) 2021-02-20T11:12:29.140 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.575093296225511) > weight (0.25) 2021-02-20T11:12:29.140 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.754740222928818) > weight (0.25) 2021-02-20T11:12:29.141 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T11:13:17.917 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4083767879951) > weight (0.25) 2021-02-20T11:13:17.918 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.38726404732113306) > weight (0.25) 2021-02-20T11:13:17.919 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.30497800669165576) > weight (0.25) 2021-02-20T11:13:17.919 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.870209271966063) > weight (0.25) 2021-02-20T11:13:17.920 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T11:13:44.141 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8130681383785745) > weight (0.25) 2021-02-20T11:13:44.142 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7200593601222841) > weight (0.25) 2021-02-20T11:13:44.142 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5632395743596991) > weight (0.25) 2021-02-20T11:13:44.142 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T11:13:44.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T11:13:44.144 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T11:13:44.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T11:13:44.137+0000 7f5a8c625700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 44761) UID: 1000 2021-02-20T11:13:44.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:13:53.145 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T11:13:53.145 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:13:53.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:13:53.146 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T11:13:53.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T11:13:53.147 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T11:13:53.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T11:13:53.150 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T11:15:07.921 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T11:15:07.923 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T11:15:07.924 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T11:15:07.925 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T11:15:07.920+0000 7fe93b95a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 44208) UID: 1000 2021-02-20T11:15:08.125 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:15:13.925 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T11:15:13.925 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:15:13.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:15:13.926 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T11:15:13.926 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T11:15:13.927 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T11:15:13.930 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T11:15:13.930 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T11:15:20.151 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4033124530047325) > weight (0.25) 2021-02-20T11:15:20.152 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2557445664500225) > weight (0.25) 2021-02-20T11:15:20.152 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9176992040108319) > weight (0.25) 2021-02-20T11:15:20.152 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9135995504772777) > weight (0.25) 2021-02-20T11:15:20.153 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T11:15:37.646 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T11:15:37.638+0000 7fb533c45700 -1 reset not still connected to 0x558b8e822820 2021-02-20T11:15:37.646 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T11:15:37.638+0000 7fb533c45700 -1 reset not still connected to 0x558b8e824490 2021-02-20T11:15:37.647 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T11:15:37.638+0000 7fb533c45700 -1 reset not still connected to 0x558b91bf4750 2021-02-20T11:15:37.648 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T11:15:37.642+0000 7fb533c45700 -1 reset not still connected to 0x558b91bf5110 2021-02-20T11:15:37.649 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T11:15:37.642+0000 7fb533c45700 -1 reset not still connected to 0x558b91bf6d80 2021-02-20T11:15:37.650 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T11:15:37.642+0000 7fb533c45700 -1 reset not still connected to 0x558b91bf75a0 2021-02-20T11:15:37.651 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T11:15:37.642+0000 7fb533c45700 -1 reset not still connected to 0x558b94b8e0d0 2021-02-20T11:15:37.651 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T11:15:37.642+0000 7fb533c45700 -1 reset not still connected to 0x558b94b8e5b0 2021-02-20T11:15:37.651 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T11:15:37.642+0000 7fb533c45700 -1 reset not still connected to 0x558b94b8e8f0 2021-02-20T11:15:37.651 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T11:15:37.642+0000 7fb533c45700 -1 reset not still connected to 0x558b94b8f040 2021-02-20T11:15:37.652 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T11:15:37.642+0000 7fb533c45700 -1 reset not still connected to 0x558b9a1275f0 2021-02-20T11:15:37.652 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T11:15:37.642+0000 7fb533c45700 -1 reset not still connected to 0x558b9a1283c0 2021-02-20T11:16:15.931 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7007657894068008) > weight (0.25) 2021-02-20T11:16:15.932 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5310352812100411) > weight (0.25) 2021-02-20T11:16:15.932 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7331660838531562) > weight (0.25) 2021-02-20T11:16:15.932 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T11:16:15.933 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T11:16:15.933 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T11:16:15.935 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T11:16:15.927+0000 7f7b75ea6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 43935) UID: 1000 2021-02-20T11:16:16.335 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:16:22.934 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T11:16:22.935 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:16:22.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:16:22.935 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T11:16:22.936 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T11:16:22.936 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T11:16:22.939 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T11:16:22.939 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T11:16:50.154 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7451364934663164) > weight (0.25) 2021-02-20T11:16:50.155 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6134141672420681) > weight (0.25) 2021-02-20T11:16:50.155 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.46167731724728756) > weight (0.25) 2021-02-20T11:16:50.155 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.49462894456411033) > weight (0.25) 2021-02-20T11:16:50.155 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T11:18:01.157 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5803017686762714) > weight (0.25) 2021-02-20T11:18:01.157 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T11:18:01.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T11:18:01.159 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T11:18:01.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T11:18:01.153+0000 7f41090b0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 44396) UID: 1000 2021-02-20T11:18:01.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:18:02.939 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5777793881532204) > weight (0.25) 2021-02-20T11:18:02.940 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7764749302845863) > weight (0.25) 2021-02-20T11:18:02.940 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T11:18:02.941 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T11:18:02.941 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T11:18:02.942 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:18:02.937+0000 7f4379108700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 44860) UID: 1000 2021-02-20T11:18:03.143 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:18:03.160 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T11:18:03.160 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:18:03.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:18:03.161 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T11:18:03.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T11:18:03.162 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T11:18:03.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T11:18:03.198 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T11:18:10.942 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T11:18:10.942 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:18:10.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:18:10.943 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T11:18:10.943 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T11:18:10.944 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T11:18:10.946 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T11:18:10.947 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T11:19:26.948 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3733222102446755) > weight (0.25) 2021-02-20T11:19:26.949 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5949167176527836) > weight (0.25) 2021-02-20T11:19:26.949 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T11:19:26.950 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T11:19:26.950 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T11:19:26.951 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:19:26.943+0000 7fe0f2156700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 45322) UID: 1000 2021-02-20T11:19:27.152 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:19:33.951 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T11:19:33.951 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:19:33.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:19:33.952 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T11:19:33.952 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T11:19:33.953 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T11:19:33.956 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T11:19:33.956 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T11:19:42.199 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5181685447108423) > weight (0.25) 2021-02-20T11:19:42.199 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7137111872639819) > weight (0.25) 2021-02-20T11:19:42.200 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5751815172326625) > weight (0.25) 2021-02-20T11:19:42.200 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9191470199743464) > weight (0.25) 2021-02-20T11:19:42.200 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T11:20:42.202 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T11:20:42.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T11:20:42.204 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T11:20:42.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T11:20:42.198+0000 7f217c602700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 44578) UID: 1000 2021-02-20T11:20:42.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:20:47.205 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T11:20:47.206 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:20:47.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:20:47.206 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T11:20:47.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T11:20:47.207 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T11:20:47.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T11:20:47.211 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T11:21:17.957 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T11:21:17.958 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T11:21:17.958 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T11:21:17.964 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T11:21:17.954+0000 7fc6a32a7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 45046) UID: 1000 2021-02-20T11:21:18.364 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:21:19.959 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T11:21:19.959 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:21:19.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:21:19.960 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T11:21:19.960 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T11:21:19.960 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T11:21:19.964 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T11:21:19.964 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T11:22:19.212 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.46503946830800336) > weight (0.25) 2021-02-20T11:22:19.213 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.42965598400542737) > weight (0.25) 2021-02-20T11:22:19.213 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.26041936272012967) > weight (0.25) 2021-02-20T11:22:19.213 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T11:22:19.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T11:22:19.214 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T11:22:19.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T11:22:19.205+0000 7faf0ee9e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 44952) UID: 1000 2021-02-20T11:22:19.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:22:23.223 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T11:22:23.212+0000 7f7308f93700 -1 reset not still connected to 0x55aabd66d810 2021-02-20T11:22:23.223 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T11:22:23.212+0000 7f7308f93700 -1 reset not still connected to 0x55aac22c5110 2021-02-20T11:22:26.215 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T11:22:26.216 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:22:26.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:22:26.217 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T11:22:26.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T11:22:26.217 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T11:22:26.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T11:22:26.220 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T11:22:42.965 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8994476070211422) > weight (0.25) 2021-02-20T11:22:42.966 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5387934744928532) > weight (0.25) 2021-02-20T11:22:42.966 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T11:22:42.967 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T11:22:42.967 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T11:22:42.968 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:22:42.960+0000 7f6ac4a42700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 45414) UID: 1000 2021-02-20T11:22:43.168 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:22:44.968 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T11:22:44.968 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:22:44.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:22:44.969 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T11:22:44.969 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T11:22:44.970 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T11:22:44.972 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T11:22:44.973 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T11:24:01.974 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5903043797412932) > weight (0.25) 2021-02-20T11:24:01.975 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T11:24:01.976 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T11:24:01.976 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T11:24:01.978 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T11:24:01.971+0000 7f4eaec18700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 43753) UID: 1000 2021-02-20T11:24:02.177 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:24:06.977 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T11:24:06.978 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:24:06.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:24:06.978 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T11:24:06.979 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T11:24:06.979 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T11:24:06.987 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T11:24:06.988 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T11:24:16.221 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6963766008661239) > weight (0.25) 2021-02-20T11:24:16.221 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.434440227220492) > weight (0.25) 2021-02-20T11:24:16.221 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5073948456176779) > weight (0.25) 2021-02-20T11:24:16.222 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5753790266520524) > weight (0.25) 2021-02-20T11:24:16.222 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T11:25:37.223 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7047648757426803) > weight (0.25) 2021-02-20T11:25:37.223 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6829547926280178) > weight (0.25) 2021-02-20T11:25:37.224 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6821136895209012) > weight (0.25) 2021-02-20T11:25:37.224 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.34228477865802287) > weight (0.25) 2021-02-20T11:25:37.224 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T11:25:42.989 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5401685758860564) > weight (0.25) 2021-02-20T11:25:42.990 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6099358412401106) > weight (0.25) 2021-02-20T11:25:42.990 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T11:25:42.991 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T11:25:42.991 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T11:25:42.991 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T11:25:42.992 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:25:42.992 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:25:42.985+0000 7f668c24d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 45780) UID: 1000 2021-02-20T11:25:43.192 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:25:49.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:25:49.094 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T11:25:49.094 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T11:25:49.095 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T11:25:49.097 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T11:25:49.097 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T11:26:51.098 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T11:26:51.100 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T11:26:51.100 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T11:26:51.101 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T11:26:51.101 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:26:51.102 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T11:26:51.096+0000 7f67d197f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 45597) UID: 1000 2021-02-20T11:26:51.302 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:26:57.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:26:57.203 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T11:26:57.204 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T11:26:57.204 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T11:26:57.207 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T11:26:57.207 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T11:27:27.226 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T11:27:27.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T11:27:27.227 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T11:27:27.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T11:27:27.219+0000 7efc044ae700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 45506) UID: 1000 2021-02-20T11:27:27.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:27:34.228 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T11:27:34.229 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:27:34.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:27:34.229 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T11:27:34.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T11:27:34.230 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T11:27:34.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T11:27:34.233 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T11:28:02.210 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.723685393432037) > weight (0.25) 2021-02-20T11:28:02.211 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8050494522689658) > weight (0.25) 2021-02-20T11:28:02.211 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.40438674639203276) > weight (0.25) 2021-02-20T11:28:02.211 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3254402778741806) > weight (0.25) 2021-02-20T11:28:02.211 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T11:29:16.235 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.499859479537477) > weight (0.25) 2021-02-20T11:29:16.235 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5945651644514295) > weight (0.25) 2021-02-20T11:29:16.235 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6522183588911519) > weight (0.25) 2021-02-20T11:29:16.236 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T11:29:16.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T11:29:16.237 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T11:29:16.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T11:29:16.229+0000 7fb243275700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 45689) UID: 1000 2021-02-20T11:29:16.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:29:19.238 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T11:29:19.239 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:29:19.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:29:19.239 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T11:29:19.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T11:29:19.240 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T11:29:19.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T11:29:19.243 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T11:29:22.212 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9951782333824313) > weight (0.25) 2021-02-20T11:29:22.213 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T11:29:22.213 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T11:29:22.214 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T11:29:22.214 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T11:29:22.205+0000 7fb5dcf8a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 45871) UID: 1000 2021-02-20T11:29:22.415 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:29:23.215 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T11:29:23.215 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:29:23.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:29:23.216 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T11:29:23.216 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T11:29:23.217 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T11:29:23.219 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T11:29:23.219 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T11:30:27.244 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T11:30:27.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T11:30:27.246 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T11:30:27.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T11:30:27.240+0000 7f57e62af700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 46147) UID: 1000 2021-02-20T11:30:27.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:30:31.246 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T11:30:31.247 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:30:31.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:30:31.248 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T11:30:31.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T11:30:31.248 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T11:30:31.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T11:30:31.251 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T11:31:05.220 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.43925129050803713) > weight (0.25) 2021-02-20T11:31:05.220 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8144583688476723) > weight (0.25) 2021-02-20T11:31:05.221 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8782329238623027) > weight (0.25) 2021-02-20T11:31:05.221 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6902569407091939) > weight (0.25) 2021-02-20T11:31:05.221 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T11:31:51.252 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T11:31:51.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T11:31:51.254 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T11:31:51.254 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T11:31:51.254 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:31:51.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T11:31:51.247+0000 7febea554700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 46421) UID: 1000 2021-02-20T11:31:51.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:31:57.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:31:57.357 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T11:31:57.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T11:31:57.358 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T11:31:57.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T11:31:57.363 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T11:32:37.222 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7675814346266303) > weight (0.25) 2021-02-20T11:32:37.223 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5489238488460683) > weight (0.25) 2021-02-20T11:32:37.223 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9808965083033913) > weight (0.25) 2021-02-20T11:32:37.223 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4482696101233852) > weight (0.25) 2021-02-20T11:32:37.223 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T11:33:26.364 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8042793882542252) > weight (0.25) 2021-02-20T11:33:26.365 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6505739558863877) > weight (0.25) 2021-02-20T11:33:26.365 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T11:33:26.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T11:33:26.366 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T11:33:26.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T11:33:26.357+0000 7f89974e7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 44487) UID: 1000 2021-02-20T11:33:26.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:33:28.368 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T11:33:28.368 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:33:28.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:33:28.369 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T11:33:28.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T11:33:28.370 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T11:33:28.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T11:33:28.373 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T11:34:28.225 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T11:34:28.226 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T11:34:28.226 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T11:34:28.227 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T11:34:28.220+0000 7f2db9078700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 46056) UID: 1000 2021-02-20T11:34:28.428 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:34:32.228 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T11:34:32.228 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:34:32.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:34:32.229 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T11:34:32.229 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T11:34:32.230 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T11:34:32.233 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T11:34:32.233 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T11:34:56.373 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6132808678860705) > weight (0.25) 2021-02-20T11:34:56.374 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4130766968273001) > weight (0.25) 2021-02-20T11:34:56.374 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7930081553090795) > weight (0.25) 2021-02-20T11:34:56.375 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7257780764217097) > weight (0.25) 2021-02-20T11:34:56.375 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T11:35:59.376 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T11:35:59.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T11:35:59.377 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T11:35:59.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T11:35:59.371+0000 7f37fa78d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 46513) UID: 1000 2021-02-20T11:35:59.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:36:06.379 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T11:36:06.379 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:36:06.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:36:06.380 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T11:36:06.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T11:36:06.381 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T11:36:06.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T11:36:06.384 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T11:36:08.234 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6952074849577885) > weight (0.25) 2021-02-20T11:36:08.234 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7485253666208505) > weight (0.25) 2021-02-20T11:36:08.235 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T11:36:08.235 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T11:36:08.235 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T11:36:08.236 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:36:08.226+0000 7f536c656700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 45965) UID: 1000 2021-02-20T11:36:08.437 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:36:11.236 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T11:36:11.236 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:36:11.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:36:11.237 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T11:36:11.237 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T11:36:11.238 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T11:36:11.240 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T11:36:11.241 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T11:37:49.385 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T11:37:49.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T11:37:49.387 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T11:37:49.387 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T11:37:49.388 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:37:49.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T11:37:49.377+0000 7f8726711700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 46797) UID: 1000 2021-02-20T11:37:49.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:37:52.242 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T11:37:52.242 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T11:37:52.243 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T11:37:52.244 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T11:37:52.233+0000 7f009afb0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 46703) UID: 1000 2021-02-20T11:37:52.444 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:37:55.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:37:55.490 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T11:37:55.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T11:37:55.492 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T11:37:55.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T11:37:55.496 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T11:38:01.244 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T11:38:01.244 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:38:01.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:38:01.245 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T11:38:01.246 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T11:38:01.246 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T11:38:01.249 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T11:38:01.250 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T11:39:36.251 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T11:39:36.253 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T11:39:36.253 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T11:39:36.255 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T11:39:36.247+0000 7f319b8b3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 47070) UID: 1000 2021-02-20T11:39:36.454 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:39:37.254 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T11:39:37.255 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:39:37.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:39:37.255 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T11:39:37.256 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T11:39:37.256 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T11:39:37.259 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T11:39:37.260 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T11:39:51.499 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7281712577918581) > weight (0.25) 2021-02-20T11:39:51.499 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T11:39:51.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T11:39:51.500 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T11:39:51.500 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T11:39:51.500 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:39:51.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T11:39:51.491+0000 7f0f352aa700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 45231) UID: 1000 2021-02-20T11:39:51.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:39:57.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:39:57.602 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T11:39:57.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T11:39:57.603 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T11:39:57.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T11:39:57.606 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T11:40:23.694 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T11:40:23.686+0000 7fb533c45700 -1 reset not still connected to 0x558b8e4da270 2021-02-20T11:40:37.261 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4855179412302617) > weight (0.25) 2021-02-20T11:40:37.262 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T11:40:37.262 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T11:40:37.262 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T11:40:37.263 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T11:40:37.258+0000 7f93728a9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 46329) UID: 1000 2021-02-20T11:40:37.464 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:40:40.264 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T11:40:40.264 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:40:40.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:40:40.265 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T11:40:40.265 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T11:40:40.266 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T11:40:40.269 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T11:40:40.269 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T11:41:49.270 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4808062744961239) > weight (0.25) 2021-02-20T11:41:49.270 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6996481581283044) > weight (0.25) 2021-02-20T11:41:49.271 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T11:41:49.271 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T11:41:49.272 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T11:41:49.272 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:41:49.261+0000 7f92fa9f7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 46888) UID: 1000 2021-02-20T11:41:49.473 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:41:49.606 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.696252016282053) > weight (0.25) 2021-02-20T11:41:49.607 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.568711294109531) > weight (0.25) 2021-02-20T11:41:49.607 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T11:41:49.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T11:41:49.608 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T11:41:49.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T11:41:49.601+0000 7fc986e2d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 46604) UID: 1000 2021-02-20T11:41:49.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:41:52.273 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T11:41:52.273 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:41:52.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:41:52.274 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T11:41:52.274 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T11:41:52.275 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T11:41:52.280 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T11:41:52.281 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T11:41:54.608 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T11:41:54.609 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:41:54.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:41:54.610 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T11:41:54.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T11:41:54.610 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T11:41:54.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T11:41:54.613 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T11:43:31.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.967143800668754) > weight (0.25) 2021-02-20T11:43:31.283 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T11:43:31.284 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T11:43:31.284 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T11:43:31.285 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T11:43:31.275+0000 7efda71ca700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 47352) UID: 1000 2021-02-20T11:43:31.485 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:43:34.614 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T11:43:34.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T11:43:34.615 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T11:43:34.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T11:43:34.607+0000 7ff65ca8e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 46979) UID: 1000 2021-02-20T11:43:34.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:43:37.285 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T11:43:37.286 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:43:37.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:43:37.287 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T11:43:37.287 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T11:43:37.287 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T11:43:37.290 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T11:43:37.290 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T11:43:42.615 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T11:43:42.615 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:43:42.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:43:42.616 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T11:43:42.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T11:43:42.617 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T11:43:42.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T11:43:42.620 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T11:44:49.417 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T11:44:49.406+0000 7f7308f93700 -1 reset not still connected to 0x55aac22c7260 2021-02-20T11:45:11.292 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.43542032827997346) > weight (0.25) 2021-02-20T11:45:11.292 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4708340217758419) > weight (0.25) 2021-02-20T11:45:11.292 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.88943613232788) > weight (0.25) 2021-02-20T11:45:11.293 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7418791472703564) > weight (0.25) 2021-02-20T11:45:11.293 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T11:45:39.621 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6373573474297166) > weight (0.25) 2021-02-20T11:45:39.622 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T11:45:39.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T11:45:39.623 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T11:45:39.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T11:45:39.617+0000 7fe975e67700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 47260) UID: 1000 2021-02-20T11:45:39.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:45:48.624 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T11:45:48.625 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:45:48.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:45:48.626 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T11:45:48.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T11:45:48.626 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T11:45:48.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T11:45:48.630 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T11:46:42.294 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.34021278824667756) > weight (0.25) 2021-02-20T11:46:42.295 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9435165858675046) > weight (0.25) 2021-02-20T11:46:42.295 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7282022380646964) > weight (0.25) 2021-02-20T11:46:42.295 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8917992754069602) > weight (0.25) 2021-02-20T11:46:42.295 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T11:47:41.631 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5229169321661585) > weight (0.25) 2021-02-20T11:47:41.631 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4447804711006712) > weight (0.25) 2021-02-20T11:47:41.632 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6207622596014422) > weight (0.25) 2021-02-20T11:47:41.632 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9979428035131199) > weight (0.25) 2021-02-20T11:47:41.632 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T11:48:30.296 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8027378227599619) > weight (0.25) 2021-02-20T11:48:30.297 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.46470135329414275) > weight (0.25) 2021-02-20T11:48:30.298 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2837557870883036) > weight (0.25) 2021-02-20T11:48:30.298 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6289607817921036) > weight (0.25) 2021-02-20T11:48:30.298 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T11:49:35.634 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8292155334141024) > weight (0.25) 2021-02-20T11:49:35.635 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7474496381704551) > weight (0.25) 2021-02-20T11:49:35.635 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8711130361994082) > weight (0.25) 2021-02-20T11:49:35.635 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.844574816739779) > weight (0.25) 2021-02-20T11:49:35.636 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T11:50:14.299 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6829569621144832) > weight (0.25) 2021-02-20T11:50:14.300 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9876454774583089) > weight (0.25) 2021-02-20T11:50:14.300 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T11:50:14.301 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T11:50:14.302 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T11:50:14.303 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:50:14.296+0000 7f634e3d2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 47443) UID: 1000 2021-02-20T11:50:14.503 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:50:17.303 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T11:50:17.303 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:50:17.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:50:17.308 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T11:50:17.308 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T11:50:17.309 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T11:50:17.312 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T11:50:17.313 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T11:51:09.637 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5986051960985397) > weight (0.25) 2021-02-20T11:51:09.638 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T11:51:09.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T11:51:09.639 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T11:51:09.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T11:51:09.631+0000 7f8da51f3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 47877) UID: 1000 2021-02-20T11:51:09.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:51:15.640 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T11:51:15.641 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:51:15.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:51:15.642 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T11:51:15.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T11:51:15.642 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T11:51:15.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T11:51:15.645 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T11:51:20.314 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.43183295194888416) > weight (0.25) 2021-02-20T11:51:20.314 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.851305177034639) > weight (0.25) 2021-02-20T11:51:20.314 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.42786799129825814) > weight (0.25) 2021-02-20T11:51:20.315 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.31016217886898334) > weight (0.25) 2021-02-20T11:51:20.315 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T11:52:59.646 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6180293410195498) > weight (0.25) 2021-02-20T11:52:59.647 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6027301609629893) > weight (0.25) 2021-02-20T11:52:59.647 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.25285386963955425) > weight (0.25) 2021-02-20T11:52:59.647 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.39631562888731997) > weight (0.25) 2021-02-20T11:52:59.647 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T11:53:09.316 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T11:53:09.317 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T11:53:09.317 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T11:53:09.318 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T11:53:09.309+0000 7fe2ecb5c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 47161) UID: 1000 2021-02-20T11:53:09.519 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:53:13.318 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T11:53:13.319 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:53:13.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:53:13.319 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T11:53:13.320 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T11:53:13.320 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T11:53:13.323 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T11:53:13.323 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T11:54:24.648 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8189496502798237) > weight (0.25) 2021-02-20T11:54:24.649 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9012210156516235) > weight (0.25) 2021-02-20T11:54:24.649 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.270855257302271) > weight (0.25) 2021-02-20T11:54:24.649 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T11:54:24.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T11:54:24.651 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T11:54:24.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T11:54:24.644+0000 7f060ef2d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 46238) UID: 1000 2021-02-20T11:54:25.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:54:28.651 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T11:54:28.652 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:54:28.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:54:28.653 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T11:54:28.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T11:54:28.654 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T11:54:28.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T11:54:28.657 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T11:54:43.323 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5203929377232456) > weight (0.25) 2021-02-20T11:54:43.324 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T11:54:43.325 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T11:54:43.325 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T11:54:43.326 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T11:54:43.315+0000 7f1780807700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 47692) UID: 1000 2021-02-20T11:54:43.526 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:54:45.849 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T11:54:45.835+0000 7f7308f93700 -1 reset not still connected to 0x55aac7f89190 2021-02-20T11:54:45.849 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T11:54:45.835+0000 7f7308f93700 -1 reset not still connected to 0x55aac7f89330 2021-02-20T11:54:45.849 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T11:54:45.835+0000 7f7308f93700 -1 reset not still connected to 0x55aac8dd79b0 2021-02-20T11:54:45.850 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T11:54:45.839+0000 7f7308f93700 -1 reset not still connected to 0x55aacd8bf860 2021-02-20T11:54:45.973 INFO:tasks.ceph.cluster2.osd.0.smithi090.stderr:2021-02-20T11:54:45.963+0000 7f7bd2757700 -1 reset not still connected to 0x5624cd5c1450 2021-02-20T11:54:45.974 INFO:tasks.ceph.cluster2.osd.0.smithi090.stderr:2021-02-20T11:54:45.963+0000 7f7bd2757700 -1 reset not still connected to 0x5624cd5c2080 2021-02-20T11:54:50.326 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T11:54:50.326 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:54:50.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:54:50.327 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T11:54:50.327 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T11:54:50.328 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T11:54:50.330 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T11:54:50.331 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T11:55:50.658 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T11:55:50.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T11:55:50.659 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T11:55:50.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T11:55:50.650+0000 7ff80f549700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 47783) UID: 1000 2021-02-20T11:55:50.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:55:55.660 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T11:55:55.660 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:55:55.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:55:55.661 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T11:55:55.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T11:55:55.662 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T11:55:55.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T11:55:55.665 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T11:56:42.333 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.43851261384907525) > weight (0.25) 2021-02-20T11:56:42.334 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5396757343376414) > weight (0.25) 2021-02-20T11:56:42.334 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T11:56:42.335 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T11:56:42.335 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T11:56:42.336 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T11:56:42.329+0000 7fadcd933700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 47969) UID: 1000 2021-02-20T11:56:42.539 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:56:43.336 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T11:56:43.337 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:56:43.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:56:43.337 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T11:56:43.338 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T11:56:43.338 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T11:56:43.341 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T11:56:43.341 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T11:57:08.666 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.46450178957850585) > weight (0.25) 2021-02-20T11:57:08.667 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T11:57:08.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T11:57:08.668 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T11:57:08.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T11:57:08.657+0000 7f3a465d7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 48060) UID: 1000 2021-02-20T11:57:09.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:57:13.668 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T11:57:13.669 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:57:13.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:57:13.669 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T11:57:13.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T11:57:13.670 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T11:57:13.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T11:57:13.673 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T11:58:18.342 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.587950905217607) > weight (0.25) 2021-02-20T11:58:18.342 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T11:58:18.343 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T11:58:18.344 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T11:58:18.344 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T11:58:18.336+0000 7f07a9cbd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 48341) UID: 1000 2021-02-20T11:58:18.545 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:58:23.345 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T11:58:23.345 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:58:23.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:58:23.346 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T11:58:23.346 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T11:58:23.347 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T11:58:23.350 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T11:58:23.350 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T11:58:28.675 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9758056357157895) > weight (0.25) 2021-02-20T11:58:28.675 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T11:58:28.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T11:58:28.676 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T11:58:28.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T11:58:28.668+0000 7f3b2222d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 48625) UID: 1000 2021-02-20T11:58:28.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:58:33.677 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T11:58:33.677 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:58:33.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:58:33.678 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T11:58:33.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T11:58:33.678 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T11:58:33.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T11:58:33.681 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T11:59:39.352 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7646178918848869) > weight (0.25) 2021-02-20T11:59:39.352 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4964397708149265) > weight (0.25) 2021-02-20T11:59:39.352 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4998081216226514) > weight (0.25) 2021-02-20T11:59:39.353 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T11:59:39.353 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T11:59:39.354 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T11:59:39.355 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T11:59:39.346+0000 7fee32062700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 45137) UID: 1000 2021-02-20T11:59:39.755 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:59:40.354 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T11:59:40.355 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:59:40.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:59:40.355 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T11:59:40.356 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T11:59:40.356 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T11:59:40.359 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T11:59:40.360 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T12:00:01.683 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9480995708222619) > weight (0.25) 2021-02-20T12:00:01.683 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.29383461054687177) > weight (0.25) 2021-02-20T12:00:01.684 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7638022948820629) > weight (0.25) 2021-02-20T12:00:01.684 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9475684841757567) > weight (0.25) 2021-02-20T12:00:01.684 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T12:01:03.361 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:01:03.363 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:01:03.363 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T12:01:03.364 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:01:03.364 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:01:03.365 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T12:01:03.357+0000 7f4b7c0b8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 48151) UID: 1000 2021-02-20T12:01:03.565 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:01:09.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:01:09.467 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:01:09.467 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:01:09.468 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:01:09.471 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:01:09.471 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T12:01:17.685 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.30572664239427594) > weight (0.25) 2021-02-20T12:01:17.686 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9643398469660964) > weight (0.25) 2021-02-20T12:01:17.686 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5721415524254035) > weight (0.25) 2021-02-20T12:01:17.686 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7430495287604248) > weight (0.25) 2021-02-20T12:01:17.687 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T12:02:30.687 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6876787898682757) > weight (0.25) 2021-02-20T12:02:30.688 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9633868789056057) > weight (0.25) 2021-02-20T12:02:30.688 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9218418909173873) > weight (0.25) 2021-02-20T12:02:30.688 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6716631381657382) > weight (0.25) 2021-02-20T12:02:30.688 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T12:02:58.472 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9518001415600239) > weight (0.25) 2021-02-20T12:02:58.472 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T12:02:58.473 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T12:02:58.474 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T12:02:58.475 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T12:02:58.467+0000 7fdd0dbc3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 48716) UID: 1000 2021-02-20T12:02:58.875 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:03:05.475 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T12:03:05.475 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:03:05.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:03:05.476 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T12:03:05.476 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T12:03:05.477 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T12:03:05.479 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T12:03:05.479 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T12:03:52.689 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6559488605189691) > weight (0.25) 2021-02-20T12:03:52.690 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8604221861679419) > weight (0.25) 2021-02-20T12:03:52.690 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T12:03:52.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T12:03:52.691 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T12:03:52.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T12:03:52.682+0000 7fecbc7c2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 47534) UID: 1000 2021-02-20T12:03:52.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:03:56.692 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T12:03:56.692 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:03:56.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:03:56.693 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T12:03:56.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T12:03:56.694 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T12:03:56.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T12:03:56.698 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T12:04:05.480 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:04:05.481 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:04:05.481 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T12:04:05.482 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T12:04:05.474+0000 7f4f9632a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 48990) UID: 1000 2021-02-20T12:04:05.883 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:04:12.482 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:04:12.482 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:04:12.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:04:12.483 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:04:12.483 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:04:12.484 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:04:12.486 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:04:12.487 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T12:05:12.699 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7873647775985089) > weight (0.25) 2021-02-20T12:05:12.700 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5757796136633185) > weight (0.25) 2021-02-20T12:05:12.700 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7894683441373337) > weight (0.25) 2021-02-20T12:05:12.700 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7042730031058387) > weight (0.25) 2021-02-20T12:05:12.700 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T12:05:55.488 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4896671294117486) > weight (0.25) 2021-02-20T12:05:55.488 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9267900442747515) > weight (0.25) 2021-02-20T12:05:55.488 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8383633751573195) > weight (0.25) 2021-02-20T12:05:55.489 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.49694192673646975) > weight (0.25) 2021-02-20T12:05:55.489 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T12:06:46.701 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:06:46.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:06:46.703 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T12:06:46.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T12:06:46.699+0000 7f5b2bba8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 48435) UID: 1000 2021-02-20T12:06:46.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:06:48.704 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:06:48.704 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:06:48.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:06:48.705 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:06:48.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:06:48.706 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:06:48.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:06:48.709 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T12:07:26.490 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5757231078959489) > weight (0.25) 2021-02-20T12:07:26.491 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2860801562575026) > weight (0.25) 2021-02-20T12:07:26.491 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8855914380497659) > weight (0.25) 2021-02-20T12:07:26.491 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9179757219367396) > weight (0.25) 2021-02-20T12:07:26.492 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T12:08:41.710 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4831549008596726) > weight (0.25) 2021-02-20T12:08:41.711 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7069888207476341) > weight (0.25) 2021-02-20T12:08:41.711 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9942857163626146) > weight (0.25) 2021-02-20T12:08:41.711 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.39929501485659213) > weight (0.25) 2021-02-20T12:08:41.711 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T12:09:17.492 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.33406009704099926) > weight (0.25) 2021-02-20T12:09:17.493 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T12:09:17.493 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T12:09:17.494 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T12:09:17.495 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T12:09:17.489+0000 7fe01bd80700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 49081) UID: 1000 2021-02-20T12:09:17.895 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:09:24.495 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T12:09:24.495 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:09:24.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:09:24.496 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T12:09:24.496 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T12:09:24.497 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T12:09:24.499 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T12:09:24.500 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T12:10:25.712 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:10:25.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:10:25.714 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T12:10:25.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T12:10:25.711+0000 7f406cb7b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 49359) UID: 1000 2021-02-20T12:10:25.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:10:29.714 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:10:29.715 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:10:29.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:10:29.716 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:10:29.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:10:29.716 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:10:29.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:10:29.720 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T12:10:33.500 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:10:33.501 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:10:33.501 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T12:10:33.502 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T12:10:33.495+0000 7f0fc9a7c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 49265) UID: 1000 2021-02-20T12:10:33.903 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:10:40.503 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:10:40.503 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:10:40.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:10:40.504 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:10:40.504 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:10:40.504 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:10:40.507 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:10:40.507 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T12:11:39.721 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:11:39.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:11:39.722 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T12:11:39.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T12:11:39.718+0000 7feec6578700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 49543) UID: 1000 2021-02-20T12:11:39.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:11:48.723 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:11:48.723 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:11:48.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:11:48.724 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:11:48.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:11:48.725 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:11:48.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:11:48.728 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T12:12:37.509 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.27268142218900693) > weight (0.25) 2021-02-20T12:12:37.509 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5471396604761078) > weight (0.25) 2021-02-20T12:12:37.509 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6634502486412514) > weight (0.25) 2021-02-20T12:12:37.510 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T12:12:37.510 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T12:12:37.511 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T12:12:37.512 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T12:12:37.509+0000 7f6448a5c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 48898) UID: 1000 2021-02-20T12:12:37.712 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:12:39.512 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T12:12:39.512 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:12:39.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:12:39.513 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T12:12:39.513 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T12:12:39.514 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T12:12:39.516 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T12:12:39.517 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T12:13:27.729 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3192638309002034) > weight (0.25) 2021-02-20T12:13:27.729 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T12:13:27.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T12:13:27.730 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T12:13:27.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T12:13:27.724+0000 7f35b81bc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 48807) UID: 1000 2021-02-20T12:13:27.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:13:36.732 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T12:13:36.732 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:13:36.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:13:36.733 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T12:13:36.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T12:13:36.734 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T12:13:36.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T12:13:36.737 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T12:14:31.518 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:14:31.519 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:14:31.520 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T12:14:31.522 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T12:14:31.515+0000 7fbb450e9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 49634) UID: 1000 2021-02-20T12:14:31.721 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:14:34.521 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:14:34.521 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:14:34.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:14:34.522 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:14:34.522 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:14:34.523 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:14:34.525 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:14:34.526 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T12:15:00.738 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8086847905404712) > weight (0.25) 2021-02-20T12:15:00.739 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T12:15:00.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T12:15:00.739 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T12:15:00.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T12:15:00.739+0000 7f43769f5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 49907) UID: 1000 2021-02-20T12:15:00.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:15:07.740 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T12:15:07.741 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:15:07.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:15:07.741 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T12:15:07.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T12:15:07.742 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T12:15:07.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T12:15:07.745 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T12:16:04.527 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:16:04.528 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:16:04.528 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T12:16:04.529 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T12:16:04.522+0000 7f0d0c0f3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 49998) UID: 1000 2021-02-20T12:16:04.729 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:16:11.529 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:16:11.530 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:16:11.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:16:11.531 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:16:11.531 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:16:11.531 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:16:11.534 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:16:11.534 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T12:16:32.745 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:16:32.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:16:32.746 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T12:16:32.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T12:16:32.741+0000 7fe315cc6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 49725) UID: 1000 2021-02-20T12:16:32.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:16:33.747 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:16:33.748 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:16:33.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:16:33.748 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:16:33.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:16:33.749 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:16:33.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:16:33.752 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T12:17:23.535 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:17:23.536 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:17:23.536 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T12:17:23.537 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T12:17:23.532+0000 7fd8fff57700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 50183) UID: 1000 2021-02-20T12:17:23.738 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:17:24.537 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:17:24.538 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:17:24.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:17:24.539 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:17:24.539 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:17:24.539 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:17:24.543 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:17:24.543 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T12:17:45.752 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:17:45.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:17:45.754 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T12:17:45.754 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:17:45.754 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:17:45.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T12:17:45.752+0000 7ff4b3c9e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 50275) UID: 1000 2021-02-20T12:17:45.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:17:51.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:17:51.862 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:17:51.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:17:51.862 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:17:51.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:17:51.866 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T12:19:12.544 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:19:12.545 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:19:12.545 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T12:19:12.546 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T12:19:12.542+0000 7ffa85053700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 50369) UID: 1000 2021-02-20T12:19:12.747 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:19:17.546 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:19:17.547 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:19:17.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:19:17.547 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:19:17.548 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:19:17.548 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:19:17.552 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:19:17.553 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T12:19:25.867 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8801703947423848) > weight (0.25) 2021-02-20T12:19:25.867 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.884796797561829) > weight (0.25) 2021-02-20T12:19:25.868 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5465173071688159) > weight (0.25) 2021-02-20T12:19:25.868 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.45225824843869356) > weight (0.25) 2021-02-20T12:19:25.868 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T12:20:35.869 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9341199867390767) > weight (0.25) 2021-02-20T12:20:35.869 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.49552088923053184) > weight (0.25) 2021-02-20T12:20:35.870 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T12:20:35.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T12:20:35.871 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T12:20:35.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T12:20:35.869+0000 7f3cd187d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 49174) UID: 1000 2021-02-20T12:20:36.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:20:37.872 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T12:20:37.872 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:20:37.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:20:37.873 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T12:20:37.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T12:20:37.873 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T12:20:37.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T12:20:37.877 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T12:20:38.554 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.560509515139166) > weight (0.25) 2021-02-20T12:20:38.554 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8040108173843978) > weight (0.25) 2021-02-20T12:20:38.554 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T12:20:38.555 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T12:20:38.555 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T12:20:38.556 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T12:20:38.549+0000 7ff3a92f4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 48526) UID: 1000 2021-02-20T12:20:38.957 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:20:46.557 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T12:20:46.557 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:20:46.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:20:46.558 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T12:20:46.558 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T12:20:46.558 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T12:20:46.561 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T12:20:46.561 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T12:22:07.878 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.32972741233096703) > weight (0.25) 2021-02-20T12:22:07.879 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T12:22:07.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T12:22:07.880 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T12:22:07.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T12:22:07.879+0000 7fcd9467b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 50092) UID: 1000 2021-02-20T12:22:08.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:22:14.881 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T12:22:14.881 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:22:14.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:22:14.882 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T12:22:14.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T12:22:14.883 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T12:22:14.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T12:22:14.886 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T12:22:19.562 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.37985181203543716) > weight (0.25) 2021-02-20T12:22:19.562 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.409235734171435) > weight (0.25) 2021-02-20T12:22:19.562 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4903213844045232) > weight (0.25) 2021-02-20T12:22:19.563 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.40442087631273904) > weight (0.25) 2021-02-20T12:22:19.563 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T12:23:32.887 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2871810205496439) > weight (0.25) 2021-02-20T12:23:32.887 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T12:23:32.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T12:23:32.888 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T12:23:32.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T12:23:32.886+0000 7fe484e64700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 50826) UID: 1000 2021-02-20T12:23:33.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:23:39.889 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T12:23:39.889 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:23:39.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:23:39.890 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T12:23:39.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T12:23:39.891 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T12:23:39.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T12:23:39.894 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T12:24:13.564 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6006452956766816) > weight (0.25) 2021-02-20T12:24:13.565 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3585926243699269) > weight (0.25) 2021-02-20T12:24:13.565 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5774415694297659) > weight (0.25) 2021-02-20T12:24:13.565 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5797123875810076) > weight (0.25) 2021-02-20T12:24:13.565 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T12:24:52.895 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8867166258017424) > weight (0.25) 2021-02-20T12:24:52.895 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7489022392248248) > weight (0.25) 2021-02-20T12:24:52.895 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5437227065662028) > weight (0.25) 2021-02-20T12:24:52.896 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5213867258467434) > weight (0.25) 2021-02-20T12:24:52.896 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T12:25:52.567 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9253423971674966) > weight (0.25) 2021-02-20T12:25:52.567 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.558725959196717) > weight (0.25) 2021-02-20T12:25:52.567 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9836492873801134) > weight (0.25) 2021-02-20T12:25:52.568 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8150848370349952) > weight (0.25) 2021-02-20T12:25:52.568 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T12:25:57.897 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5555734183996668) > weight (0.25) 2021-02-20T12:25:57.897 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T12:25:57.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T12:25:57.898 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T12:25:57.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T12:25:57.896+0000 7f4659236700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 50917) UID: 1000 2021-02-20T12:25:58.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:26:01.899 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T12:26:01.899 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:26:01.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:26:01.900 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T12:26:01.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T12:26:01.901 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T12:26:01.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T12:26:01.903 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T12:27:24.905 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7113704577266399) > weight (0.25) 2021-02-20T12:27:24.906 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9078610858177817) > weight (0.25) 2021-02-20T12:27:24.906 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T12:27:24.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T12:27:24.907 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T12:27:24.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T12:27:24.906+0000 7f57a31a3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 50644) UID: 1000 2021-02-20T12:27:25.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:27:27.908 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T12:27:27.908 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:27:27.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:27:27.909 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T12:27:27.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T12:27:27.909 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T12:27:27.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T12:27:27.912 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T12:27:48.568 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:27:48.569 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:27:48.569 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T12:27:48.570 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T12:27:48.570+0000 7f43d453a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 50552) UID: 1000 2021-02-20T12:27:48.771 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:27:56.571 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:27:56.571 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:27:56.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:27:56.572 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:27:56.572 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:27:56.572 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:27:56.576 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:27:56.577 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T12:29:09.914 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:29:09.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:29:09.915 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T12:29:09.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T12:29:09.912+0000 7f2006406700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 50460) UID: 1000 2021-02-20T12:29:10.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:29:16.917 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:29:16.917 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:29:16.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:29:16.919 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:29:16.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:29:16.920 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:29:16.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:29:16.925 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T12:29:22.578 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.33856280209382417) > weight (0.25) 2021-02-20T12:29:22.578 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T12:29:22.579 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T12:29:22.579 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T12:29:22.580 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T12:29:22.580+0000 7f67c3714700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 49451) UID: 1000 2021-02-20T12:29:22.981 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:29:29.580 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T12:29:29.581 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:29:29.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:29:29.582 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T12:29:29.582 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T12:29:29.582 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T12:29:29.585 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T12:29:29.585 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T12:30:16.925 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9570533936490561) > weight (0.25) 2021-02-20T12:30:16.926 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T12:30:16.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T12:30:16.927 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T12:30:16.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T12:30:16.927+0000 7fe760572700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 51012) UID: 1000 2021-02-20T12:30:17.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:30:20.927 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T12:30:20.928 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:30:20.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:30:20.929 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T12:30:20.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T12:30:20.929 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T12:30:20.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T12:30:20.933 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T12:30:39.770 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T12:30:39.767+0000 7f7308f93700 -1 reset not still connected to 0x55aac2357c70 2021-02-20T12:30:54.586 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8160204638191404) > weight (0.25) 2021-02-20T12:30:54.586 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7374391079565321) > weight (0.25) 2021-02-20T12:30:54.587 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6111266115826843) > weight (0.25) 2021-02-20T12:30:54.587 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T12:30:54.588 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T12:30:54.588 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T12:30:54.589 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T12:30:54.587+0000 7f2c7b148700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 49816) UID: 1000 2021-02-20T12:30:54.989 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:30:59.591 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T12:30:59.591 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:30:59.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:30:59.592 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T12:30:59.593 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T12:30:59.593 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T12:30:59.596 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T12:30:59.596 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T12:32:14.597 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.902159020436977) > weight (0.25) 2021-02-20T12:32:14.598 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.25583592870698135) > weight (0.25) 2021-02-20T12:32:14.598 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5769658002472476) > weight (0.25) 2021-02-20T12:32:14.598 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T12:32:14.599 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T12:32:14.599 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T12:32:14.601 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T12:32:14.597+0000 7f61f7e7c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 51560) UID: 1000 2021-02-20T12:32:14.801 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:32:16.933 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5614429762611436) > weight (0.25) 2021-02-20T12:32:16.934 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8942077383968946) > weight (0.25) 2021-02-20T12:32:16.934 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T12:32:16.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T12:32:16.935 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T12:32:16.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T12:32:16.933+0000 7f6bd0bb1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 51103) UID: 1000 2021-02-20T12:32:17.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:32:17.936 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T12:32:17.936 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:32:17.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:32:17.937 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T12:32:17.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T12:32:17.937 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T12:32:17.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T12:32:17.940 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T12:32:19.600 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T12:32:19.600 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:32:19.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:32:19.601 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T12:32:19.601 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T12:32:19.601 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T12:32:19.604 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T12:32:19.604 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T12:33:25.605 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.42790955483986026) > weight (0.25) 2021-02-20T12:33:25.606 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8665473273057942) > weight (0.25) 2021-02-20T12:33:25.606 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5816982327300088) > weight (0.25) 2021-02-20T12:33:25.606 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5590877630661554) > weight (0.25) 2021-02-20T12:33:25.606 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T12:33:52.941 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:33:52.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:33:52.943 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T12:33:52.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T12:33:52.943+0000 7fa2bfdf7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 51285) UID: 1000 2021-02-20T12:33:53.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:33:59.944 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:33:59.945 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:33:59.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:33:59.946 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:33:59.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:33:59.946 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:33:59.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:33:59.949 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T12:35:05.608 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.41573644054385406) > weight (0.25) 2021-02-20T12:35:05.609 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.39846612559468597) > weight (0.25) 2021-02-20T12:35:05.609 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T12:35:05.610 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T12:35:05.611 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T12:35:05.611 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T12:35:05.610+0000 7f049bc9f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 50735) UID: 1000 2021-02-20T12:35:06.012 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:35:11.612 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T12:35:11.612 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:35:11.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:35:11.613 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T12:35:11.613 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T12:35:11.614 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T12:35:11.616 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T12:35:11.617 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T12:35:23.950 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:35:23.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:35:23.951 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T12:35:23.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T12:35:23.950+0000 7f0af2223700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 51834) UID: 1000 2021-02-20T12:35:24.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:35:30.952 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:35:30.953 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:35:30.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:35:30.953 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:35:30.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:35:30.954 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:35:30.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:35:30.957 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T12:36:45.958 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.25987185841246885) > weight (0.25) 2021-02-20T12:36:45.959 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.46779437029436555) > weight (0.25) 2021-02-20T12:36:45.959 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6191468607198617) > weight (0.25) 2021-02-20T12:36:45.960 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.25936601004068116) > weight (0.25) 2021-02-20T12:36:45.960 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T12:37:06.618 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6081570756329094) > weight (0.25) 2021-02-20T12:37:06.618 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2758263936002413) > weight (0.25) 2021-02-20T12:37:06.618 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T12:37:06.619 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T12:37:06.619 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T12:37:06.620 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T12:37:06.616+0000 7f5576b5f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 51928) UID: 1000 2021-02-20T12:37:06.821 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:37:09.620 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T12:37:09.621 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:37:09.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:37:09.621 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T12:37:09.622 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T12:37:09.622 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T12:37:09.625 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T12:37:09.625 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T12:37:50.962 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.884424859070512) > weight (0.25) 2021-02-20T12:37:50.963 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4950498435066705) > weight (0.25) 2021-02-20T12:37:50.963 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3613485078272861) > weight (0.25) 2021-02-20T12:37:50.963 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T12:37:50.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T12:37:50.964 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T12:37:50.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T12:37:50.963+0000 7fd9e04b6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 48242) UID: 1000 2021-02-20T12:37:51.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:37:55.965 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T12:37:55.965 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:37:55.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:37:55.966 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T12:37:55.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T12:37:55.967 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T12:37:55.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T12:37:55.970 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T12:38:38.626 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6872161300897899) > weight (0.25) 2021-02-20T12:38:38.626 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T12:38:38.627 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T12:38:38.627 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T12:38:38.628 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T12:38:38.627+0000 7fbdeb4b4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 51376) UID: 1000 2021-02-20T12:38:38.828 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:38:47.629 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T12:38:47.630 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:38:47.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:38:47.631 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T12:38:47.631 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T12:38:47.631 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T12:38:47.636 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T12:38:47.636 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T12:38:57.971 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7037258570855328) > weight (0.25) 2021-02-20T12:38:57.971 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2663224048748478) > weight (0.25) 2021-02-20T12:38:57.971 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T12:38:57.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T12:38:57.972 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T12:38:57.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T12:38:57.970+0000 7fde46754700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 51652) UID: 1000 2021-02-20T12:38:58.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:39:04.973 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T12:39:04.974 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:39:04.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:39:04.975 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T12:39:04.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T12:39:04.976 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T12:39:04.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T12:39:04.987 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T12:40:32.637 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8794690933799068) > weight (0.25) 2021-02-20T12:40:32.639 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7763013776316497) > weight (0.25) 2021-02-20T12:40:32.640 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7612141449367771) > weight (0.25) 2021-02-20T12:40:32.640 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.333199892209632) > weight (0.25) 2021-02-20T12:40:32.641 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T12:40:58.987 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:40:58.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:40:58.989 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T12:40:58.989 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:40:58.989 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:40:58.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T12:40:58.984+0000 7f5c5f7e7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 52019) UID: 1000 2021-02-20T12:40:59.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:41:05.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:41:05.091 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:41:05.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:41:05.092 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:41:05.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:41:05.094 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T12:41:42.643 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.573459926675473) > weight (0.25) 2021-02-20T12:41:42.643 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T12:41:42.644 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T12:41:42.644 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T12:41:42.645 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T12:41:42.644+0000 7fbc4ef48700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 52300) UID: 1000 2021-02-20T12:41:42.846 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:41:48.645 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T12:41:48.645 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:41:48.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:41:48.646 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T12:41:48.646 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T12:41:48.647 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T12:41:48.649 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T12:41:48.650 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T12:42:38.095 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.49277400941250027) > weight (0.25) 2021-02-20T12:42:38.096 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5167643232343705) > weight (0.25) 2021-02-20T12:42:38.096 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4944566494463113) > weight (0.25) 2021-02-20T12:42:38.096 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7468077273254873) > weight (0.25) 2021-02-20T12:42:38.097 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T12:42:48.651 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4366047006447856) > weight (0.25) 2021-02-20T12:42:48.652 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T12:42:48.652 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T12:42:48.652 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T12:42:48.653 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T12:42:48.650+0000 7f8cd2367700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 52575) UID: 1000 2021-02-20T12:42:48.854 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:42:50.653 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T12:42:50.654 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:42:50.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:42:50.655 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T12:42:50.656 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T12:42:50.656 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T12:42:50.660 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T12:42:50.660 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T12:44:07.662 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7166463675579396) > weight (0.25) 2021-02-20T12:44:07.663 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.44933373798806775) > weight (0.25) 2021-02-20T12:44:07.663 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T12:44:07.664 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T12:44:07.664 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T12:44:07.666 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T12:44:07.661+0000 7f4452419700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 52110) UID: 1000 2021-02-20T12:44:07.865 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:44:15.666 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T12:44:15.667 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:44:15.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:44:15.668 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T12:44:15.668 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T12:44:15.668 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T12:44:15.673 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T12:44:15.674 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T12:44:23.097 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.48211208798481286) > weight (0.25) 2021-02-20T12:44:23.098 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6213500082783425) > weight (0.25) 2021-02-20T12:44:23.098 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T12:44:23.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T12:44:23.099 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T12:44:23.099 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T12:44:23.099 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:44:23.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T12:44:23.097+0000 7fd2c3e40700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 52392) UID: 1000 2021-02-20T12:44:23.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:44:29.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:44:29.201 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T12:44:29.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T12:44:29.202 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T12:44:29.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T12:44:29.205 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T12:46:06.675 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7416009507679489) > weight (0.25) 2021-02-20T12:46:06.675 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.36193223732832824) > weight (0.25) 2021-02-20T12:46:06.676 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.659018044306859) > weight (0.25) 2021-02-20T12:46:06.676 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5204826284306809) > weight (0.25) 2021-02-20T12:46:06.676 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T12:46:07.206 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:46:07.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:46:07.207 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T12:46:07.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T12:46:07.207+0000 7f345cd41700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 52484) UID: 1000 2021-02-20T12:46:07.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:46:09.208 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:46:09.208 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:46:09.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:46:09.209 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:46:09.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:46:09.210 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:46:09.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:46:09.219 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T12:47:06.678 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:47:06.679 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:47:06.679 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T12:47:06.680 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T12:47:06.674+0000 7f9c146a6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 51194) UID: 1000 2021-02-20T12:47:07.081 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:47:08.680 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:47:08.681 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:47:08.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:47:08.682 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:47:08.683 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:47:08.683 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:47:08.687 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:47:08.687 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T12:47:50.221 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.39397823950646826) > weight (0.25) 2021-02-20T12:47:50.222 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8856608167386704) > weight (0.25) 2021-02-20T12:47:50.222 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T12:47:50.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T12:47:50.223 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T12:47:50.224 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T12:47:50.224 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:47:50.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T12:47:50.221+0000 7fa91a483700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 52848) UID: 1000 2021-02-20T12:47:50.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:47:56.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:47:56.327 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T12:47:56.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T12:47:56.327 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T12:47:56.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T12:47:56.330 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T12:48:24.688 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:48:24.689 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:48:24.690 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T12:48:24.691 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T12:48:24.685+0000 7f0b768af700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 53033) UID: 1000 2021-02-20T12:48:24.891 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:48:32.691 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:48:32.691 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:48:32.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:48:32.692 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:48:32.692 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:48:32.692 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:48:32.695 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:48:32.695 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T12:49:26.331 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7330387092261067) > weight (0.25) 2021-02-20T12:49:26.332 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T12:49:26.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T12:49:26.333 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T12:49:26.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T12:49:26.332+0000 7f3e6f7fb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 51469) UID: 1000 2021-02-20T12:49:26.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:49:30.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T12:49:30.644+0000 7f542a3d2700 -1 rbd::mirror::ImageReplayer: 0x556eed97a000 [3/c64947f6-af7d-4dad-b5bd-4c3ed0c6fde9] start: already running 2021-02-20T12:49:30.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T12:49:30.644+0000 7f542abd3700 -1 rbd::mirror::ImageReplayer: 0x556eed97a280 [3/c82eb13a-1d88-4f85-a9e5-d88cf352a989] start: already running 2021-02-20T12:49:32.335 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T12:49:32.335 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:49:32.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:49:32.336 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T12:49:32.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T12:49:32.336 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T12:49:32.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T12:49:32.339 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T12:49:32.697 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.46045368101237005) > weight (0.25) 2021-02-20T12:49:32.697 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6031194124097824) > weight (0.25) 2021-02-20T12:49:32.697 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T12:49:32.698 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T12:49:32.698 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T12:49:32.699 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T12:49:32.695+0000 7fc2505d0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 52757) UID: 1000 2021-02-20T12:49:32.899 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:49:41.699 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T12:49:41.700 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:49:41.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:49:41.701 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T12:49:41.701 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T12:49:41.702 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T12:49:41.707 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T12:49:41.708 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T12:50:45.340 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4275375927828786) > weight (0.25) 2021-02-20T12:50:45.341 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6962074237079223) > weight (0.25) 2021-02-20T12:50:45.341 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.598240412076001) > weight (0.25) 2021-02-20T12:50:45.341 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4362243108065983) > weight (0.25) 2021-02-20T12:50:45.341 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T12:51:25.710 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.936223556850377) > weight (0.25) 2021-02-20T12:51:25.711 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5222627048506618) > weight (0.25) 2021-02-20T12:51:25.711 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5999879194900746) > weight (0.25) 2021-02-20T12:51:25.712 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8831688784191214) > weight (0.25) 2021-02-20T12:51:25.712 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T12:51:57.342 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5955113270364886) > weight (0.25) 2021-02-20T12:51:57.343 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6353803385125829) > weight (0.25) 2021-02-20T12:51:57.344 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T12:51:57.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T12:51:57.346 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T12:51:57.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T12:51:57.341+0000 7f5439972700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 53124) UID: 1000 2021-02-20T12:51:57.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:52:04.347 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T12:52:04.348 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:52:04.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:52:04.349 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T12:52:04.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T12:52:04.349 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T12:52:04.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T12:52:04.353 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T12:52:49.713 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.650424926707153) > weight (0.25) 2021-02-20T12:52:49.730 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T12:52:49.730 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T12:52:49.731 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T12:52:49.731 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T12:52:49.728+0000 7f7981fd2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 52666) UID: 1000 2021-02-20T12:52:50.132 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:52:57.732 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T12:52:57.739 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:52:57.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:52:57.740 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T12:52:57.741 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T12:52:57.741 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T12:52:57.744 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T12:52:57.745 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T12:53:45.355 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.27250414863652894) > weight (0.25) 2021-02-20T12:53:45.356 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7695020612778037) > weight (0.25) 2021-02-20T12:53:45.356 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6046790545064185) > weight (0.25) 2021-02-20T12:53:45.356 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.663649554765342) > weight (0.25) 2021-02-20T12:53:45.356 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T12:54:04.746 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:54:04.754 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:54:04.755 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T12:54:04.755 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T12:54:04.751+0000 7f4db6609700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 53215) UID: 1000 2021-02-20T12:54:04.956 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:54:10.755 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:54:10.759 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:54:10.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:54:10.760 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:54:10.760 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:54:10.761 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:54:10.764 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:54:10.764 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T12:55:13.357 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:55:13.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:55:13.359 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T12:55:13.359 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:55:13.359 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:55:13.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T12:55:13.358+0000 7f0d688c6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 52942) UID: 1000 2021-02-20T12:55:13.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:55:19.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:55:19.462 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:55:19.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:55:19.462 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:55:19.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:55:19.465 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T12:55:31.766 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2988604504407609) > weight (0.25) 2021-02-20T12:55:31.766 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.43002360847309473) > weight (0.25) 2021-02-20T12:55:31.766 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5015350165718104) > weight (0.25) 2021-02-20T12:55:31.766 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5076060215221859) > weight (0.25) 2021-02-20T12:55:31.767 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T12:56:26.467 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5279402306922754) > weight (0.25) 2021-02-20T12:56:26.467 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2743502982732645) > weight (0.25) 2021-02-20T12:56:26.468 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8695797021748459) > weight (0.25) 2021-02-20T12:56:26.468 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T12:56:26.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T12:56:26.469 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T12:56:26.469 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T12:56:26.469 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:56:26.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T12:56:26.464+0000 7f542b3d4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 52209) UID: 1000 2021-02-20T12:56:26.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:56:32.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:56:32.690 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T12:56:32.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T12:56:32.691 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T12:56:32.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T12:56:32.697 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T12:57:07.768 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8394568302895136) > weight (0.25) 2021-02-20T12:57:07.768 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7643751801302197) > weight (0.25) 2021-02-20T12:57:07.768 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.33856782857034284) > weight (0.25) 2021-02-20T12:57:07.769 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.50244944036713) > weight (0.25) 2021-02-20T12:57:07.769 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T12:57:44.697 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7293997076600364) > weight (0.25) 2021-02-20T12:57:44.704 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4101270511306334) > weight (0.25) 2021-02-20T12:57:44.705 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5260536381550187) > weight (0.25) 2021-02-20T12:57:44.705 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7421526341920813) > weight (0.25) 2021-02-20T12:57:44.705 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T12:58:40.770 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4387855555231863) > weight (0.25) 2021-02-20T12:58:40.777 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.355921030384482) > weight (0.25) 2021-02-20T12:58:40.778 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T12:58:40.778 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T12:58:40.779 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T12:58:40.779 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T12:58:40.779 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:58:40.780 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T12:58:40.778+0000 7f531cdb9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 53405) UID: 1000 2021-02-20T12:58:41.180 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:58:46.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:58:46.881 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T12:58:46.881 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T12:58:46.882 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T12:58:46.884 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T12:58:46.885 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T12:59:25.706 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:59:25.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:59:25.708 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T12:59:25.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T12:59:25.705+0000 7f2cb500f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 53789) UID: 1000 2021-02-20T12:59:25.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:59:32.708 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:59:32.709 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:59:32.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:59:32.710 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:59:32.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:59:32.710 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:59:32.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:59:32.713 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T13:00:19.886 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4504816793559051) > weight (0.25) 2021-02-20T13:00:19.887 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8356071049718067) > weight (0.25) 2021-02-20T13:00:19.887 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.37757848744721234) > weight (0.25) 2021-02-20T13:00:19.888 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9642125613109269) > weight (0.25) 2021-02-20T13:00:19.888 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T13:01:27.716 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4145742947247216) > weight (0.25) 2021-02-20T13:01:27.749 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6810374755993391) > weight (0.25) 2021-02-20T13:01:27.749 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7769977341095279) > weight (0.25) 2021-02-20T13:01:27.749 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7846613133842907) > weight (0.25) 2021-02-20T13:01:27.749 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T13:01:28.047 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T13:01:28.040+0000 7fb533c45700 -1 reset not still connected to 0x558b98a2ac30 2021-02-20T13:01:28.047 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T13:01:28.040+0000 7fb533c45700 -1 reset not still connected to 0x558b9a129670 2021-02-20T13:01:40.889 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8275489639517442) > weight (0.25) 2021-02-20T13:01:40.897 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7116248632826037) > weight (0.25) 2021-02-20T13:01:40.898 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5918929739027115) > weight (0.25) 2021-02-20T13:01:40.898 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3675210869967207) > weight (0.25) 2021-02-20T13:01:40.899 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T13:02:51.900 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2551922639288108) > weight (0.25) 2021-02-20T13:02:51.917 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:02:51.919 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:02:51.919 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T13:02:51.922 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T13:02:51.918+0000 7fde4cc34700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 53604) UID: 1000 2021-02-20T13:02:52.123 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:02:54.921 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T13:02:54.930 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:02:54.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:02:54.931 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T13:02:54.931 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:02:54.932 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:02:54.935 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:02:54.935 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T13:03:06.750 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3765625148167425) > weight (0.25) 2021-02-20T13:03:06.757 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9112098436191772) > weight (0.25) 2021-02-20T13:03:06.758 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T13:03:06.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T13:03:06.759 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T13:03:06.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T13:03:06.754+0000 7fd745978700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 53497) UID: 1000 2021-02-20T13:03:06.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:03:14.759 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T13:03:14.766 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:03:14.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:03:14.767 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T13:03:14.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T13:03:14.767 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T13:03:14.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T13:03:14.773 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T13:04:48.776 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.28476012986643906) > weight (0.25) 2021-02-20T13:04:48.790 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.42841925175918705) > weight (0.25) 2021-02-20T13:04:48.790 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6957165427723351) > weight (0.25) 2021-02-20T13:04:48.791 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.29398282893546157) > weight (0.25) 2021-02-20T13:04:48.791 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T13:04:50.936 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6021164973048594) > weight (0.25) 2021-02-20T13:04:50.937 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2561477094248529) > weight (0.25) 2021-02-20T13:04:50.937 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8059290268651701) > weight (0.25) 2021-02-20T13:04:50.937 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9931748077223094) > weight (0.25) 2021-02-20T13:04:50.937 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T13:06:23.939 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:06:23.968 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:06:23.968 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T13:06:23.969 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T13:06:23.966+0000 7f6ea518c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 53695) UID: 1000 2021-02-20T13:06:24.370 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:06:31.970 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:06:31.970 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:06:31.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:06:31.971 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:06:31.971 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:06:31.972 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:06:31.975 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:06:31.976 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T13:06:47.793 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:06:47.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:06:47.794 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T13:06:47.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T13:06:47.794+0000 7ffa513ef700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 54072) UID: 1000 2021-02-20T13:06:47.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:06:54.795 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:06:54.802 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:06:54.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:06:54.803 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:06:54.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:06:54.803 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:06:54.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:06:54.806 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T13:07:48.977 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4618754993425547) > weight (0.25) 2021-02-20T13:07:48.985 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8725062143028812) > weight (0.25) 2021-02-20T13:07:48.985 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9802378817415928) > weight (0.25) 2021-02-20T13:07:48.986 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3064885884814351) > weight (0.25) 2021-02-20T13:07:48.986 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T13:08:11.807 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.34831174033131507) > weight (0.25) 2021-02-20T13:08:11.815 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.667801549266719) > weight (0.25) 2021-02-20T13:08:11.815 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7123713200729035) > weight (0.25) 2021-02-20T13:08:11.816 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.46739657316805183) > weight (0.25) 2021-02-20T13:08:11.816 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T13:09:17.987 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4260900539196568) > weight (0.25) 2021-02-20T13:09:18.034 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6829906805598013) > weight (0.25) 2021-02-20T13:09:18.035 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T13:09:18.037 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T13:09:18.037 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T13:09:18.039 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T13:09:18.035+0000 7f841465a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 53980) UID: 1000 2021-02-20T13:09:18.239 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:09:24.039 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T13:09:24.046 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:09:24.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:09:24.047 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T13:09:24.047 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T13:09:24.048 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T13:09:24.050 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T13:09:24.051 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T13:09:32.818 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8020233346262863) > weight (0.25) 2021-02-20T13:09:32.825 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5834646020902707) > weight (0.25) 2021-02-20T13:09:32.825 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T13:09:32.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T13:09:32.826 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T13:09:32.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T13:09:32.823+0000 7f5504a0d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 54255) UID: 1000 2021-02-20T13:09:33.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:09:33.828 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T13:09:33.838 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:09:33.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:09:33.839 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T13:09:33.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T13:09:33.839 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T13:09:33.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T13:09:33.843 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T13:10:58.843 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5164247656469696) > weight (0.25) 2021-02-20T13:10:58.844 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5525232911562203) > weight (0.25) 2021-02-20T13:10:58.844 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9456455687457145) > weight (0.25) 2021-02-20T13:10:58.845 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.47098505456666506) > weight (0.25) 2021-02-20T13:10:58.845 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T13:11:12.051 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4202506217524501) > weight (0.25) 2021-02-20T13:11:12.058 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.32498000098880486) > weight (0.25) 2021-02-20T13:11:12.058 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9527033627851658) > weight (0.25) 2021-02-20T13:11:12.059 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.32679064932055957) > weight (0.25) 2021-02-20T13:11:12.059 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T13:12:13.846 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:12:13.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:12:13.881 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T13:12:13.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T13:12:13.880+0000 7f1398c53700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 54440) UID: 1000 2021-02-20T13:12:14.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:12:18.882 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:12:18.890 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:12:18.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:12:18.891 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:12:18.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:12:18.891 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:12:18.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:12:18.894 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T13:13:07.059 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:13:07.069 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:13:07.069 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T13:13:07.071 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T13:13:07.067+0000 7fa8dce34700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 54349) UID: 1000 2021-02-20T13:13:07.271 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:13:10.070 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:13:10.070 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:13:10.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:13:10.071 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:13:10.071 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:13:10.072 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:13:10.081 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:13:10.081 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T13:13:54.896 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3610236713433168) > weight (0.25) 2021-02-20T13:13:54.897 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5983028647868912) > weight (0.25) 2021-02-20T13:13:54.897 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8413526819046236) > weight (0.25) 2021-02-20T13:13:54.897 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3816412954891708) > weight (0.25) 2021-02-20T13:13:54.897 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T13:14:19.082 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:14:19.083 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:14:19.084 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T13:14:19.085 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T13:14:19.082+0000 7f289f733700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 54805) UID: 1000 2021-02-20T13:14:19.285 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:14:23.084 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:14:23.085 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:14:23.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:14:23.086 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:14:23.086 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:14:23.087 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:14:23.090 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:14:23.090 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T13:14:56.898 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6001446202706241) > weight (0.25) 2021-02-20T13:14:56.899 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4269067632988358) > weight (0.25) 2021-02-20T13:14:56.899 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T13:14:56.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T13:14:56.900 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T13:14:56.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T13:14:56.897+0000 7f19b9560700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 54622) UID: 1000 2021-02-20T13:14:57.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:15:02.901 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T13:15:02.913 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:15:02.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:15:02.914 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T13:15:02.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T13:15:02.915 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T13:15:02.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T13:15:02.921 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T13:15:39.091 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2593303591171887) > weight (0.25) 2021-02-20T13:15:39.099 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.40421235309226955) > weight (0.25) 2021-02-20T13:15:39.099 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9665276228101732) > weight (0.25) 2021-02-20T13:15:39.100 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.28488004072807926) > weight (0.25) 2021-02-20T13:15:39.100 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T13:16:38.922 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8549272572038271) > weight (0.25) 2021-02-20T13:16:38.929 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5941160999796686) > weight (0.25) 2021-02-20T13:16:38.929 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9051848257743246) > weight (0.25) 2021-02-20T13:16:38.930 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8102730235905548) > weight (0.25) 2021-02-20T13:16:38.930 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T13:17:14.102 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2817173674359026) > weight (0.25) 2021-02-20T13:17:14.110 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:17:14.111 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:17:14.111 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T13:17:14.113 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T13:17:14.111+0000 7fc7223c4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 54164) UID: 1000 2021-02-20T13:17:14.313 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:17:23.112 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T13:17:23.112 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:17:23.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:17:23.113 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T13:17:23.114 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:17:23.114 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:17:23.117 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:17:23.117 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T13:18:07.932 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.49027305626709616) > weight (0.25) 2021-02-20T13:18:07.932 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9454452657374013) > weight (0.25) 2021-02-20T13:18:07.933 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7075510826104502) > weight (0.25) 2021-02-20T13:18:07.933 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T13:18:07.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T13:18:07.934 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T13:18:07.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T13:18:07.930+0000 7f4f9aaf3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 53888) UID: 1000 2021-02-20T13:18:08.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:18:16.935 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T13:18:16.936 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:18:16.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:18:16.937 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T13:18:16.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T13:18:16.937 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T13:18:16.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T13:18:16.940 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T13:18:47.118 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8301106530415051) > weight (0.25) 2021-02-20T13:18:47.119 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:18:47.119 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:18:47.120 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T13:18:47.120 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T13:18:47.117+0000 7f307c989700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 55085) UID: 1000 2021-02-20T13:18:47.321 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:18:50.121 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T13:18:50.121 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:18:50.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:18:50.123 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T13:18:50.123 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:18:50.124 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:18:50.127 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:18:50.127 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T13:19:18.941 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8560817004270196) > weight (0.25) 2021-02-20T13:19:18.948 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5342273847374731) > weight (0.25) 2021-02-20T13:19:18.948 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9652801398963032) > weight (0.25) 2021-02-20T13:19:18.949 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9963072007696477) > weight (0.25) 2021-02-20T13:19:18.949 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T13:20:05.128 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3061295038776738) > weight (0.25) 2021-02-20T13:20:05.137 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3464180266679425) > weight (0.25) 2021-02-20T13:20:05.137 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2676221131202309) > weight (0.25) 2021-02-20T13:20:05.137 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7238543711352938) > weight (0.25) 2021-02-20T13:20:05.137 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T13:21:15.950 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:21:15.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:21:15.952 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T13:21:15.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T13:21:15.951+0000 7f1a88925700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 54714) UID: 1000 2021-02-20T13:21:16.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:21:17.953 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:21:17.954 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:21:17.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:21:17.954 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:21:17.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:21:17.955 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:21:17.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:21:17.963 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T13:21:36.138 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6709713249666219) > weight (0.25) 2021-02-20T13:21:36.139 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7328577337385528) > weight (0.25) 2021-02-20T13:21:36.139 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7625366035598735) > weight (0.25) 2021-02-20T13:21:36.139 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7806127271523401) > weight (0.25) 2021-02-20T13:21:36.140 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T13:22:17.964 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9489934403873856) > weight (0.25) 2021-02-20T13:22:17.979 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T13:22:17.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T13:22:17.980 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T13:22:17.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T13:22:17.978+0000 7f14c78d7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 53314) UID: 1000 2021-02-20T13:22:18.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:22:19.981 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T13:22:19.981 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:22:19.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:22:19.982 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T13:22:19.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T13:22:19.982 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T13:22:19.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T13:22:19.985 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T13:22:44.141 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8464131263372344) > weight (0.25) 2021-02-20T13:22:44.155 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7482916468066787) > weight (0.25) 2021-02-20T13:22:44.155 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7357693449064865) > weight (0.25) 2021-02-20T13:22:44.155 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4758114435253088) > weight (0.25) 2021-02-20T13:22:44.156 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T13:23:48.987 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:23:49.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:23:49.002 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T13:23:49.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T13:23:49.000+0000 7fa2f8ab6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 55359) UID: 1000 2021-02-20T13:23:49.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:23:53.002 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:23:53.003 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:23:53.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:23:53.004 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:23:53.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:23:53.005 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:23:53.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:23:53.009 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T13:23:55.157 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8053960457154593) > weight (0.25) 2021-02-20T13:23:55.157 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6769908764538869) > weight (0.25) 2021-02-20T13:23:55.157 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7246239933898099) > weight (0.25) 2021-02-20T13:23:55.157 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.98996084654038) > weight (0.25) 2021-02-20T13:23:55.158 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T13:24:57.010 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2938627298980483) > weight (0.25) 2021-02-20T13:24:57.018 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3599418394434969) > weight (0.25) 2021-02-20T13:24:57.019 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7169161419194917) > weight (0.25) 2021-02-20T13:24:57.019 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5867858096299979) > weight (0.25) 2021-02-20T13:24:57.019 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T13:25:38.158 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5048608265655622) > weight (0.25) 2021-02-20T13:25:38.170 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7672947428383937) > weight (0.25) 2021-02-20T13:25:38.170 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8543975746121678) > weight (0.25) 2021-02-20T13:25:38.170 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5344572836973719) > weight (0.25) 2021-02-20T13:25:38.170 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T13:26:28.020 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.35929772136232296) > weight (0.25) 2021-02-20T13:26:28.028 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T13:26:28.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T13:26:28.030 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T13:26:28.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T13:26:28.029+0000 7f9903e68700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 55451) UID: 1000 2021-02-20T13:26:28.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:26:31.031 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T13:26:31.040 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:26:31.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:26:31.041 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T13:26:31.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T13:26:31.043 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T13:26:31.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T13:26:31.048 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T13:26:46.175 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9589179494443832) > weight (0.25) 2021-02-20T13:26:46.175 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7861633200759852) > weight (0.25) 2021-02-20T13:26:46.175 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8843521731872258) > weight (0.25) 2021-02-20T13:26:46.176 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2680161180043037) > weight (0.25) 2021-02-20T13:26:46.176 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T13:27:33.049 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3676635427072841) > weight (0.25) 2021-02-20T13:27:33.057 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6489038849385971) > weight (0.25) 2021-02-20T13:27:33.057 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8505869547112384) > weight (0.25) 2021-02-20T13:27:33.058 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6659198753691832) > weight (0.25) 2021-02-20T13:27:33.058 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T13:28:00.177 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7635740743769688) > weight (0.25) 2021-02-20T13:28:00.201 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:28:00.202 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:28:00.203 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T13:28:00.204 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T13:28:00.204+0000 7f65d3fb3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 55267) UID: 1000 2021-02-20T13:28:00.404 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:28:09.204 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T13:28:09.205 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:28:09.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:28:09.205 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T13:28:09.206 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:28:09.206 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:28:09.209 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:28:09.210 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T13:28:35.059 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:28:35.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:28:35.061 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T13:28:35.061 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:28:35.061 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:28:35.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T13:28:35.059+0000 7fe0a86d8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 55550) UID: 1000 2021-02-20T13:28:35.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:28:41.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:28:41.164 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:28:41.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:28:41.165 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:28:41.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:28:41.168 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T13:29:32.210 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9416067867731046) > weight (0.25) 2021-02-20T13:29:32.217 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:29:32.218 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:29:32.219 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T13:29:32.219 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T13:29:32.214+0000 7fa359d69700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 55735) UID: 1000 2021-02-20T13:29:32.420 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:29:40.219 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T13:29:40.220 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:29:40.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:29:40.221 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T13:29:40.221 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:29:40.222 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:29:40.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:29:40.225 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T13:30:12.169 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3545153781203261) > weight (0.25) 2021-02-20T13:30:12.170 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4437947086222269) > weight (0.25) 2021-02-20T13:30:12.170 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T13:30:12.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T13:30:12.171 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T13:30:12.171 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T13:30:12.172 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:30:12.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T13:30:12.170+0000 7fc3899ab700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 54990) UID: 1000 2021-02-20T13:30:12.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:30:18.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:30:18.274 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T13:30:18.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T13:30:18.274 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T13:30:18.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T13:30:18.278 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T13:31:16.226 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:31:16.236 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:31:16.236 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T13:31:16.238 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T13:31:16.233+0000 7fa01f360700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 54896) UID: 1000 2021-02-20T13:31:16.437 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:31:22.238 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:31:22.239 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:31:22.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:31:22.239 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:31:22.240 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:31:22.240 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:31:22.245 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:31:22.246 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T13:31:50.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2955996618491401) > weight (0.25) 2021-02-20T13:31:50.287 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T13:31:50.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T13:31:50.288 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T13:31:50.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T13:31:50.288+0000 7f2be44b5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 55644) UID: 1000 2021-02-20T13:31:50.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:31:59.290 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T13:31:59.296 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:31:59.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:31:59.297 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T13:31:59.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T13:31:59.298 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T13:31:59.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T13:31:59.301 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T13:32:50.246 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5119849275975394) > weight (0.25) 2021-02-20T13:32:50.256 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7354060312734002) > weight (0.25) 2021-02-20T13:32:50.257 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.529614783892724) > weight (0.25) 2021-02-20T13:32:50.257 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T13:32:50.258 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T13:32:50.258 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T13:32:50.261 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T13:32:50.259+0000 7f3aaec48700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 51743) UID: 1000 2021-02-20T13:32:50.660 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:32:51.259 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T13:32:51.260 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:32:51.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:32:51.260 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T13:32:51.261 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T13:32:51.261 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T13:32:51.264 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T13:32:51.264 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T13:33:41.380 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T13:33:41.374+0000 7f7308f93700 -1 reset not still connected to 0x55aabadb95a0 2021-02-20T13:33:41.381 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T13:33:41.378+0000 7f7308f93700 -1 reset not still connected to 0x55aac0012d80 2021-02-20T13:33:41.382 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T13:33:41.378+0000 7f7308f93700 -1 reset not still connected to 0x55aaca8f60d0 2021-02-20T13:33:41.382 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T13:33:41.378+0000 7f7308f93700 -1 reset not still connected to 0x55aaca8f65b0 2021-02-20T13:33:41.383 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T13:33:41.378+0000 7f7308f93700 -1 reset not still connected to 0x55aaca8f7520 2021-02-20T13:33:41.383 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T13:33:41.378+0000 7f7308f93700 -1 reset not still connected to 0x55aaca8f9670 2021-02-20T13:33:41.383 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T13:33:41.378+0000 7f7308f93700 -1 reset not still connected to 0x55aae5d97450 2021-02-20T13:33:41.383 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T13:33:41.378+0000 7f7308f93700 -1 reset not still connected to 0x55aae5d976c0 2021-02-20T13:33:41.384 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T13:33:41.378+0000 7f7308f93700 -1 reset not still connected to 0x55aae5d987d0 2021-02-20T13:33:57.302 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:33:57.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:33:57.313 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T13:33:57.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T13:33:57.314+0000 7f4ea4e18700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 55826) UID: 1000 2021-02-20T13:33:57.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:33:59.314 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:33:59.314 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:33:59.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:33:59.315 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:33:59.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:33:59.316 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:33:59.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:33:59.320 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T13:34:01.265 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:34:01.271 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:34:01.271 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T13:34:01.272 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T13:34:01.270+0000 7fd557271700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 56108) UID: 1000 2021-02-20T13:34:01.472 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:34:10.272 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:34:10.273 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:34:10.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:34:10.274 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:34:10.274 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:34:10.274 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:34:10.278 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:34:10.278 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T13:35:32.321 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.30943342572408794) > weight (0.25) 2021-02-20T13:35:32.331 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5974588057095741) > weight (0.25) 2021-02-20T13:35:32.332 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.76026339421015) > weight (0.25) 2021-02-20T13:35:32.332 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T13:35:32.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T13:35:32.336 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T13:35:32.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T13:35:32.332+0000 7fc0c3f5d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 55176) UID: 1000 2021-02-20T13:35:32.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:35:37.338 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T13:35:37.343 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:35:37.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:35:37.344 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T13:35:37.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T13:35:37.344 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T13:35:37.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T13:35:37.348 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T13:35:40.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.81850662352783) > weight (0.25) 2021-02-20T13:35:40.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3455889417376625) > weight (0.25) 2021-02-20T13:35:40.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8327572883904106) > weight (0.25) 2021-02-20T13:35:40.280 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8238221005771744) > weight (0.25) 2021-02-20T13:35:40.280 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T13:37:12.349 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:37:12.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:37:12.359 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T13:37:12.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T13:37:12.358+0000 7f7ad935e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 56384) UID: 1000 2021-02-20T13:37:12.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:37:14.360 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:37:14.361 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:37:14.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:37:14.362 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:37:14.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:37:14.362 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:37:14.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:37:14.365 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T13:37:20.281 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:37:20.282 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:37:20.282 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T13:37:20.283 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T13:37:20.282+0000 7f63c6c21700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 56483) UID: 1000 2021-02-20T13:37:20.484 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:37:25.284 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:37:25.284 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:37:25.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:37:25.285 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:37:25.286 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:37:25.286 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:37:25.289 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:37:25.289 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T13:38:26.366 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8120017109144316) > weight (0.25) 2021-02-20T13:38:26.367 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T13:38:26.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T13:38:26.368 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T13:38:26.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T13:38:26.369+0000 7fe9c29b3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 56199) UID: 1000 2021-02-20T13:38:26.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:38:30.737 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T13:38:30.729+0000 7f7308f93700 -1 reset not still connected to 0x55aae5d982f0 2021-02-20T13:38:34.369 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T13:38:34.369 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:38:34.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:38:34.370 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T13:38:34.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T13:38:34.371 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T13:38:34.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T13:38:34.374 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T13:38:50.290 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:38:50.291 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:38:50.292 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T13:38:50.293 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T13:38:50.289+0000 7f7ed9636700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 56759) UID: 1000 2021-02-20T13:38:50.493 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:38:59.293 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:38:59.294 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:38:59.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:38:59.294 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:38:59.295 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:38:59.295 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:38:59.298 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:38:59.298 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T13:40:04.376 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7131628792919096) > weight (0.25) 2021-02-20T13:40:04.376 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7230302910777908) > weight (0.25) 2021-02-20T13:40:04.377 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7419376556711565) > weight (0.25) 2021-02-20T13:40:04.377 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9450162093011765) > weight (0.25) 2021-02-20T13:40:04.377 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T13:40:25.299 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.43842329639298194) > weight (0.25) 2021-02-20T13:40:25.302 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9256913906533412) > weight (0.25) 2021-02-20T13:40:25.302 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3136313068274228) > weight (0.25) 2021-02-20T13:40:25.302 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5635556912085706) > weight (0.25) 2021-02-20T13:40:25.303 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T13:41:49.378 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:41:49.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:41:49.388 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T13:41:49.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T13:41:49.386+0000 7f25b2237700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 56668) UID: 1000 2021-02-20T13:41:49.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:41:51.388 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:41:51.389 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:41:51.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:41:51.390 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:41:51.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:41:51.390 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:41:51.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:41:51.394 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T13:42:15.304 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8152677517395155) > weight (0.25) 2021-02-20T13:42:15.305 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6714989747419408) > weight (0.25) 2021-02-20T13:42:15.305 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5912152826223519) > weight (0.25) 2021-02-20T13:42:15.305 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T13:42:15.306 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T13:42:15.306 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T13:42:15.307 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T13:42:15.305+0000 7fabd4855700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 56291) UID: 1000 2021-02-20T13:42:15.508 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:42:21.307 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T13:42:21.307 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:42:21.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:42:21.308 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T13:42:21.309 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T13:42:21.309 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T13:42:21.312 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T13:42:21.312 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T13:43:10.395 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5042574936848228) > weight (0.25) 2021-02-20T13:43:10.395 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T13:43:10.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T13:43:10.396 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T13:43:10.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T13:43:10.396+0000 7f408dc9d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 56858) UID: 1000 2021-02-20T13:43:10.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:43:14.397 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T13:43:14.397 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:43:14.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:43:14.398 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T13:43:14.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T13:43:14.399 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T13:43:14.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T13:43:14.403 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T13:44:13.315 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.41492514249947776) > weight (0.25) 2021-02-20T13:44:13.328 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:44:13.329 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:44:13.329 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T13:44:13.331 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T13:44:13.327+0000 7f5a7395e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 55917) UID: 1000 2021-02-20T13:44:13.731 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:44:14.330 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T13:44:14.330 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:44:14.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:44:14.331 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T13:44:14.331 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:44:14.332 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:44:14.335 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:44:14.336 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T13:44:15.404 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5885867102474737) > weight (0.25) 2021-02-20T13:44:15.405 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8939649027598656) > weight (0.25) 2021-02-20T13:44:15.405 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T13:44:15.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T13:44:15.407 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T13:44:15.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T13:44:15.407+0000 7f75d0649700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 56017) UID: 1000 2021-02-20T13:44:15.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:44:19.408 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T13:44:19.429 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:44:19.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:44:19.431 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T13:44:19.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T13:44:19.432 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T13:44:19.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T13:44:19.438 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T13:44:19.824 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T13:44:19.819+0000 7f7308f93700 -1 reset not still connected to 0x55aac3f83450 2021-02-20T13:44:19.824 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T13:44:19.819+0000 7f7308f93700 -1 reset not still connected to 0x55aac3f85190 2021-02-20T13:44:19.825 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T13:44:19.819+0000 7f7308f93700 -1 reset not still connected to 0x55aacd8be820 2021-02-20T13:45:26.439 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:45:26.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:45:26.449 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T13:45:26.450 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:45:26.450 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:45:26.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T13:45:26.446+0000 7fe27d6f0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 57041) UID: 1000 2021-02-20T13:45:26.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:45:32.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:45:32.553 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:45:32.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:45:32.554 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:45:32.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:45:32.557 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T13:45:49.337 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:45:49.338 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:45:49.339 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T13:45:49.340 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T13:45:49.338+0000 7fe183c25700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 56949) UID: 1000 2021-02-20T13:45:49.540 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:45:55.340 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:45:55.340 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:45:55.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:45:55.341 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:45:55.341 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:45:55.341 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:45:55.344 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:45:55.344 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T13:47:18.558 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2505537359222102) > weight (0.25) 2021-02-20T13:47:18.559 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8179694949787162) > weight (0.25) 2021-02-20T13:47:18.559 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8500282450683576) > weight (0.25) 2021-02-20T13:47:18.559 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4702041513808347) > weight (0.25) 2021-02-20T13:47:18.559 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T13:47:36.345 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7882595681127095) > weight (0.25) 2021-02-20T13:47:36.346 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7141145852082147) > weight (0.25) 2021-02-20T13:47:36.346 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T13:47:36.347 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T13:47:36.347 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T13:47:36.348 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T13:47:36.348+0000 7f9e847d6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 54531) UID: 1000 2021-02-20T13:47:36.749 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:47:40.349 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T13:47:40.349 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:47:40.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:47:40.350 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T13:47:40.350 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T13:47:40.351 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T13:47:40.354 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T13:47:40.354 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T13:47:40.594 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T13:47:40.592+0000 7f685684c700 -1 rbd::mirror::ImageReplayer: 0x55976fdb8300 [3/3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd] start: already running 2021-02-20T13:48:44.561 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.48387305412603965) > weight (0.25) 2021-02-20T13:48:44.565 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6477360566013509) > weight (0.25) 2021-02-20T13:48:44.566 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7218400114573295) > weight (0.25) 2021-02-20T13:48:44.566 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T13:48:44.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T13:48:44.567 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T13:48:44.567 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T13:48:44.568 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:48:44.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T13:48:44.567+0000 7f40ae981700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 56577) UID: 1000 2021-02-20T13:48:44.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:48:50.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:48:50.671 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T13:48:50.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T13:48:50.672 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T13:48:50.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T13:48:50.675 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T13:48:58.355 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:48:58.356 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:48:58.357 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T13:48:58.358 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T13:48:58.354+0000 7f685784e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 57674) UID: 1000 2021-02-20T13:48:58.758 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:49:06.357 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:49:06.358 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:49:06.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:49:06.359 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:49:06.359 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:49:06.359 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:49:06.362 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:49:06.363 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T13:50:03.676 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.48738538207608617) > weight (0.25) 2021-02-20T13:50:03.677 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T13:50:03.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T13:50:03.678 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T13:50:03.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T13:50:03.673+0000 7f3e3605b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 57290) UID: 1000 2021-02-20T13:50:03.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:50:11.679 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T13:50:11.680 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:50:11.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:50:11.681 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T13:50:11.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T13:50:11.681 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T13:50:11.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T13:50:11.685 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T13:50:27.363 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:50:27.364 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:50:27.365 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T13:50:27.366 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T13:50:27.365+0000 7fd7d2287700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 57949) UID: 1000 2021-02-20T13:50:27.566 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:50:32.366 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:50:32.366 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:50:32.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:50:32.367 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:50:32.367 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:50:32.368 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:50:32.370 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:50:32.370 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T13:51:49.372 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9081364138430035) > weight (0.25) 2021-02-20T13:51:49.373 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6694722200099044) > weight (0.25) 2021-02-20T13:51:49.373 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8313803708991354) > weight (0.25) 2021-02-20T13:51:49.373 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8378983135632142) > weight (0.25) 2021-02-20T13:51:49.373 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T13:52:07.686 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3509563647051457) > weight (0.25) 2021-02-20T13:52:07.686 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8199691026257121) > weight (0.25) 2021-02-20T13:52:07.687 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9003296610502477) > weight (0.25) 2021-02-20T13:52:07.687 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4987849370441917) > weight (0.25) 2021-02-20T13:52:07.687 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T13:52:54.374 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.892110753966219) > weight (0.25) 2021-02-20T13:52:54.381 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.424041191930993) > weight (0.25) 2021-02-20T13:52:54.381 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9694687240357779) > weight (0.25) 2021-02-20T13:52:54.381 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8966424269898814) > weight (0.25) 2021-02-20T13:52:54.382 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T13:53:16.693 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:53:16.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:53:16.698 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T13:53:16.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T13:53:16.698+0000 7f8da8478700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 57582) UID: 1000 2021-02-20T13:53:16.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:53:19.624 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T13:53:19.620+0000 7fb533c45700 -1 reset not still connected to 0x558b8edd7fb0 2021-02-20T13:53:19.625 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T13:53:19.620+0000 7fb533c45700 -1 reset not still connected to 0x558b8edd83c0 2021-02-20T13:53:19.626 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T13:53:19.620+0000 7fb533c45700 -1 reset not still connected to 0x558b8edd9260 2021-02-20T13:53:19.626 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T13:53:19.620+0000 7fb533c45700 -1 reset not still connected to 0x558b94b8e0d0 2021-02-20T13:53:19.626 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T13:53:19.620+0000 7fb533c45700 -1 reset not still connected to 0x558b94b8e5b0 2021-02-20T13:53:19.626 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T13:53:19.620+0000 7fb533c45700 -1 reset not still connected to 0x558b94b8e820 2021-02-20T13:53:19.627 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T13:53:19.620+0000 7fb533c45700 -1 reset not still connected to 0x558b94b90be0 2021-02-20T13:53:19.627 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T13:53:19.620+0000 7fb533c45700 -1 reset not still connected to 0x558b95d3dba0 2021-02-20T13:53:19.627 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T13:53:19.620+0000 7fb533c45700 -1 reset not still connected to 0x558b95ecb0c0 2021-02-20T13:53:19.627 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T13:53:19.620+0000 7fb533c45700 -1 reset not still connected to 0x558b9a1282f0 2021-02-20T13:53:21.699 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:53:21.699 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:53:21.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:53:21.700 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:53:21.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:53:21.701 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:53:21.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:53:21.704 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T13:54:12.383 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:54:12.384 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:54:12.384 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T13:54:12.385 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:54:12.385 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:54:12.386 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T13:54:12.385+0000 7fd944e31700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 58131) UID: 1000 2021-02-20T13:54:12.586 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:54:18.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:54:18.488 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:54:18.488 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:54:18.489 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:54:18.491 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:54:18.492 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T13:54:28.705 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:54:28.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:54:28.706 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T13:54:28.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T13:54:28.705+0000 7f89284ed700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 58222) UID: 1000 2021-02-20T13:54:28.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:54:35.707 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:54:35.707 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:54:35.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:54:35.708 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:54:35.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:54:35.709 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:54:35.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:54:35.712 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T13:55:45.493 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2632611220498142) > weight (0.25) 2021-02-20T13:55:45.503 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:55:45.504 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:55:45.504 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T13:55:45.505 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T13:55:45.503+0000 7ff238563700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 57381) UID: 1000 2021-02-20T13:55:45.706 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:55:46.505 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T13:55:46.506 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:55:46.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:55:46.507 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T13:55:46.507 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:55:46.507 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:55:46.510 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:55:46.510 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T13:55:53.713 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3555755762598093) > weight (0.25) 2021-02-20T13:55:53.713 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8796471964571833) > weight (0.25) 2021-02-20T13:55:53.714 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2691525283771009) > weight (0.25) 2021-02-20T13:55:53.714 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6960325610583401) > weight (0.25) 2021-02-20T13:55:53.714 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T13:56:48.511 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6511353331813393) > weight (0.25) 2021-02-20T13:56:48.520 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:56:48.520 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:56:48.521 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T13:56:48.521 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T13:56:48.518+0000 7f66267cc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 58498) UID: 1000 2021-02-20T13:56:48.722 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:56:50.522 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T13:56:50.522 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:56:50.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:56:50.524 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T13:56:50.524 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:56:50.525 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:56:50.529 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:56:50.530 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T13:57:01.716 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8368792597490926) > weight (0.25) 2021-02-20T13:57:01.720 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T13:57:01.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T13:57:01.721 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T13:57:01.722 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T13:57:01.722 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:57:01.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T13:57:01.718+0000 7f75cc160700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 58040) UID: 1000 2021-02-20T13:57:01.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:57:07.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:57:07.825 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T13:57:07.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T13:57:07.826 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T13:57:07.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T13:57:07.829 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T13:58:09.530 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9842404889908875) > weight (0.25) 2021-02-20T13:58:09.550 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:58:09.551 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:58:09.551 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T13:58:09.552 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T13:58:09.549+0000 7faabedf0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 58590) UID: 1000 2021-02-20T13:58:09.753 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:58:17.552 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T13:58:17.553 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:58:17.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:58:17.554 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T13:58:17.554 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:58:17.554 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:58:17.557 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:58:17.557 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T13:59:01.830 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9729953870129701) > weight (0.25) 2021-02-20T13:59:01.837 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5048975146198469) > weight (0.25) 2021-02-20T13:59:01.837 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T13:59:01.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T13:59:01.840 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T13:59:01.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T13:59:01.840+0000 7fed56651700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 57472) UID: 1000 2021-02-20T13:59:02.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:59:05.841 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T13:59:05.841 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:59:05.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:59:05.842 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T13:59:05.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T13:59:05.843 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T13:59:05.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T13:59:05.846 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T13:59:57.558 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9968335797342793) > weight (0.25) 2021-02-20T13:59:57.559 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:59:57.560 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:59:57.560 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T13:59:57.561 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T13:59:57.559+0000 7f7c91fb4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 58772) UID: 1000 2021-02-20T13:59:57.761 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:00:06.562 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:00:06.562 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:00:06.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:00:06.564 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:00:06.564 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:00:06.565 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:00:06.568 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:00:06.569 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T14:00:30.847 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:00:30.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:00:30.860 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T14:00:30.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T14:00:30.859+0000 7fc90fe44700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 58404) UID: 1000 2021-02-20T14:00:31.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:00:33.861 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:00:33.869 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:00:33.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:00:33.870 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:00:33.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:00:33.871 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:00:33.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:00:33.874 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T14:01:54.570 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8356635532690747) > weight (0.25) 2021-02-20T14:01:54.579 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5464287815689607) > weight (0.25) 2021-02-20T14:01:54.579 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7085938687698091) > weight (0.25) 2021-02-20T14:01:54.580 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.675089038503683) > weight (0.25) 2021-02-20T14:01:54.580 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T14:02:16.875 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5012455364327645) > weight (0.25) 2021-02-20T14:02:16.885 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T14:02:16.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T14:02:16.888 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T14:02:16.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T14:02:16.885+0000 7fd1d7c9b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 58681) UID: 1000 2021-02-20T14:02:17.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:02:19.888 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T14:02:19.899 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:02:19.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:02:19.900 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T14:02:19.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T14:02:19.901 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T14:02:19.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T14:02:19.911 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T14:03:46.581 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.26453007918427396) > weight (0.25) 2021-02-20T14:03:46.592 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:03:46.593 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:03:46.594 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T14:03:46.596 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T14:03:46.591+0000 7fa338268700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 58963) UID: 1000 2021-02-20T14:03:46.795 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:03:49.594 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:03:49.595 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:03:49.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:03:49.596 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:03:49.597 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:03:49.598 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:03:49.601 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:03:49.602 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T14:03:55.912 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8852221696648424) > weight (0.25) 2021-02-20T14:03:55.923 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4099374135816791) > weight (0.25) 2021-02-20T14:03:55.924 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7829480106858393) > weight (0.25) 2021-02-20T14:03:55.924 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4439105647756969) > weight (0.25) 2021-02-20T14:03:55.924 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T14:05:37.603 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:05:37.615 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:05:37.615 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T14:05:37.617 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T14:05:37.613+0000 7fbd9f431700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 58313) UID: 1000 2021-02-20T14:05:37.816 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:05:42.616 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:05:42.618 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:05:42.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:05:42.619 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:05:42.619 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:05:42.620 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:05:42.625 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:05:42.625 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T14:05:47.926 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:05:47.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:05:47.927 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T14:05:47.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T14:05:47.925+0000 7f2c77df5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 59054) UID: 1000 2021-02-20T14:05:48.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:05:53.928 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:05:53.929 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:05:53.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:05:53.930 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:05:53.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:05:53.930 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:05:53.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:05:53.934 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T14:07:40.627 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6115675413167564) > weight (0.25) 2021-02-20T14:07:40.640 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7689993124346125) > weight (0.25) 2021-02-20T14:07:40.640 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T14:07:40.641 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T14:07:40.641 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T14:07:40.642 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T14:07:40.639+0000 7fe5c1eaf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 57766) UID: 1000 2021-02-20T14:07:41.045 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:07:41.642 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T14:07:41.643 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:07:41.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:07:41.643 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T14:07:41.644 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T14:07:41.644 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T14:07:41.647 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T14:07:41.647 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T14:07:48.935 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6535548550967408) > weight (0.25) 2021-02-20T14:07:48.935 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T14:07:48.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T14:07:48.936 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T14:07:48.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T14:07:48.935+0000 7ffb91f0f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 59146) UID: 1000 2021-02-20T14:07:49.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:07:52.937 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T14:07:52.938 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:07:52.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:07:52.939 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T14:07:52.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T14:07:52.940 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T14:07:52.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T14:07:52.943 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T14:09:00.944 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:09:00.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:09:00.955 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T14:09:00.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T14:09:00.954+0000 7fc9bedc8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 59438) UID: 1000 2021-02-20T14:09:01.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:09:03.956 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:09:03.956 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:09:03.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:09:03.957 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:09:03.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:09:03.958 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:09:03.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:09:03.961 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T14:09:16.649 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.46626040160718085) > weight (0.25) 2021-02-20T14:09:16.655 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6566035009727323) > weight (0.25) 2021-02-20T14:09:16.655 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5601399947472385) > weight (0.25) 2021-02-20T14:09:16.655 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8907843041589201) > weight (0.25) 2021-02-20T14:09:16.655 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T14:10:30.962 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6704157715849635) > weight (0.25) 2021-02-20T14:10:30.963 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3537618395268486) > weight (0.25) 2021-02-20T14:10:30.963 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T14:10:30.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T14:10:30.964 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T14:10:30.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T14:10:30.964+0000 7fa905165700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 58863) UID: 1000 2021-02-20T14:10:31.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:10:39.966 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T14:10:39.966 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:10:39.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:10:39.967 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T14:10:39.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T14:10:39.968 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T14:10:39.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T14:10:39.988 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T14:10:48.656 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:10:48.657 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:10:48.657 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T14:10:48.658 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T14:10:48.656+0000 7ff98c86c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 59339) UID: 1000 2021-02-20T14:10:48.860 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:10:52.659 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:10:53.484 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:10:53.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:10:53.485 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:10:53.485 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:10:53.486 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:10:53.491 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:10:53.492 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T14:11:23.493 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T14:11:23.490+0000 7fb533c45700 -1 reset not still connected to 0x558b8e8228f0 2021-02-20T14:11:23.494 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T14:11:23.490+0000 7fb533c45700 -1 reset not still connected to 0x558b94651040 2021-02-20T14:11:23.494 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T14:11:23.490+0000 7fb533c45700 -1 reset not still connected to 0x558b948cfe90 2021-02-20T14:12:15.989 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.44142359262684916) > weight (0.25) 2021-02-20T14:12:15.990 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6917353783113139) > weight (0.25) 2021-02-20T14:12:15.990 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T14:12:15.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T14:12:15.991 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T14:12:15.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T14:12:15.990+0000 7f7ab2b14700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 59804) UID: 1000 2021-02-20T14:12:16.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:12:17.992 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T14:12:17.993 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:12:17.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:12:17.994 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T14:12:17.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T14:12:17.995 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T14:12:17.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T14:12:17.998 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T14:12:38.492 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5968444408291376) > weight (0.25) 2021-02-20T14:12:38.493 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.556625417813566) > weight (0.25) 2021-02-20T14:12:38.493 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8505428859456319) > weight (0.25) 2021-02-20T14:12:38.493 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T14:12:38.494 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T14:12:38.494 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T14:12:38.495 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T14:12:38.494+0000 7f068ab1c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 57132) UID: 1000 2021-02-20T14:12:38.895 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:12:40.495 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T14:12:40.495 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:12:40.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:12:40.496 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T14:12:40.497 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T14:12:40.497 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T14:12:40.500 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T14:12:40.500 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T14:13:17.998 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.31208851358375966) > weight (0.25) 2021-02-20T14:13:17.999 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8168061319879842) > weight (0.25) 2021-02-20T14:13:17.999 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.27825949249859816) > weight (0.25) 2021-02-20T14:13:17.999 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.49737237664120415) > weight (0.25) 2021-02-20T14:13:17.999 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T14:14:35.502 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8839258528816636) > weight (0.25) 2021-02-20T14:14:35.503 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6526882449466327) > weight (0.25) 2021-02-20T14:14:35.503 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T14:14:35.504 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T14:14:35.505 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T14:14:35.506 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T14:14:35.504+0000 7f9f1efee700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 59529) UID: 1000 2021-02-20T14:14:35.706 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:14:43.506 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T14:14:43.506 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:14:43.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:14:43.507 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T14:14:43.508 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T14:14:43.508 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T14:14:43.512 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T14:14:43.512 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T14:15:16.001 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:15:16.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:15:16.003 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T14:15:16.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T14:15:16.003+0000 7f416cf6d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 59711) UID: 1000 2021-02-20T14:15:16.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:15:24.004 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:15:24.004 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:15:24.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:15:24.005 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:15:24.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:15:24.006 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:15:24.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:15:24.009 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T14:15:54.513 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9836943807356788) > weight (0.25) 2021-02-20T14:15:54.519 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5970079018052264) > weight (0.25) 2021-02-20T14:15:54.520 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8151806072107974) > weight (0.25) 2021-02-20T14:15:54.520 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.855985806796597) > weight (0.25) 2021-02-20T14:15:54.520 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T14:16:24.010 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9483087317413531) > weight (0.25) 2021-02-20T14:16:24.022 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T14:16:24.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T14:16:24.023 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T14:16:24.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T14:16:24.022+0000 7fbdb1dc6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 59620) UID: 1000 2021-02-20T14:16:24.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:16:26.025 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T14:16:26.025 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:16:26.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:16:26.026 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T14:16:26.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T14:16:26.027 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T14:16:26.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T14:16:26.030 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T14:17:20.521 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7990757951694357) > weight (0.25) 2021-02-20T14:17:20.522 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:17:20.523 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:17:20.523 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T14:17:20.524 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T14:17:20.521+0000 7facd996a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 59245) UID: 1000 2021-02-20T14:17:20.924 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:17:27.524 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:17:27.524 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:17:27.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:17:27.525 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:17:27.526 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:17:27.526 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:17:27.529 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:17:27.529 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T14:17:54.031 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6874613435608491) > weight (0.25) 2021-02-20T14:17:54.031 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6309936699291936) > weight (0.25) 2021-02-20T14:17:54.032 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6213182701503082) > weight (0.25) 2021-02-20T14:17:54.032 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6244443689316519) > weight (0.25) 2021-02-20T14:17:54.032 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T14:19:10.531 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8382939462570407) > weight (0.25) 2021-02-20T14:19:10.531 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5525297037929644) > weight (0.25) 2021-02-20T14:19:10.532 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T14:19:10.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T14:19:10.533 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T14:19:10.538 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T14:19:10.535+0000 7fce56dda700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 60169) UID: 1000 2021-02-20T14:19:10.738 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:19:19.534 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T14:19:19.535 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:19:19.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:19:19.536 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T14:19:19.537 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T14:19:19.537 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T14:19:19.552 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T14:19:19.553 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T14:19:42.034 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:19:42.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:19:42.058 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T14:19:42.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T14:19:42.055+0000 7f74ded93700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 60263) UID: 1000 2021-02-20T14:19:42.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:19:50.059 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:19:50.067 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:19:50.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:19:50.067 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:19:50.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:19:50.068 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:19:50.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:19:50.071 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T14:21:10.555 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:21:10.556 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:21:10.557 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T14:21:10.557 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T14:21:10.553+0000 7f4772213700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 59895) UID: 1000 2021-02-20T14:21:10.958 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:21:14.557 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:21:14.565 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:21:14.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:21:14.566 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:21:14.566 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:21:14.566 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:21:14.569 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:21:14.570 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T14:21:24.072 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4649726435122331) > weight (0.25) 2021-02-20T14:21:24.108 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6368018257864626) > weight (0.25) 2021-02-20T14:21:24.108 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T14:21:24.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T14:21:24.109 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T14:21:24.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T14:21:24.105+0000 7fad1fdc9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 59986) UID: 1000 2021-02-20T14:21:24.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:21:30.110 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T14:21:30.120 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:21:30.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:21:30.121 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T14:21:30.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T14:21:30.122 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T14:21:30.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T14:21:30.132 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T14:22:48.135 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5603445402277161) > weight (0.25) 2021-02-20T14:22:48.145 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4934714135242524) > weight (0.25) 2021-02-20T14:22:48.146 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.278620947527349) > weight (0.25) 2021-02-20T14:22:48.146 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T14:22:48.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T14:22:48.147 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T14:22:48.147 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T14:22:48.148 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:22:48.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T14:22:48.148+0000 7f8103dbe700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 57858) UID: 1000 2021-02-20T14:22:48.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:22:49.816 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T14:22:49.812+0000 7f7308f93700 -1 reset not still connected to 0x55aabd66ad00 2021-02-20T14:22:49.816 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T14:22:49.812+0000 7f7308f93700 -1 reset not still connected to 0x55aabd66cbe0 2021-02-20T14:22:49.824 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T14:22:49.820+0000 7f7308f93700 -1 reset not still connected to 0x55aac2357ee0 2021-02-20T14:22:49.824 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T14:22:49.820+0000 7f7308f93700 -1 reset not still connected to 0x55aac2359b50 2021-02-20T14:22:49.824 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T14:22:49.820+0000 7f7308f93700 -1 reset not still connected to 0x55aaca8f6680 2021-02-20T14:22:54.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:22:54.250 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T14:22:54.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T14:22:54.251 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T14:22:54.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T14:22:54.254 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T14:23:05.575 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7526215049075324) > weight (0.25) 2021-02-20T14:23:05.576 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.812794288743029) > weight (0.25) 2021-02-20T14:23:05.576 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5059872992214148) > weight (0.25) 2021-02-20T14:23:05.576 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7616183672299973) > weight (0.25) 2021-02-20T14:23:05.576 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T14:23:49.815 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T14:23:49.811+0000 7f7308f93700 -1 reset not still connected to 0x55aac2359b50 2021-02-20T14:23:50.016 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T14:23:49.815+0000 7f7308f93700 -1 reset not still connected to 0x55aaca8f6680 2021-02-20T14:24:19.829 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T14:24:19.824+0000 7fb533c45700 -1 reset not still connected to 0x558b948cc8f0 2021-02-20T14:24:47.255 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:24:47.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:24:47.257 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T14:24:47.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T14:24:47.254+0000 7f3826dfa700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 60630) UID: 1000 2021-02-20T14:24:47.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:24:56.258 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:24:56.269 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:24:56.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:24:56.270 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:24:56.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:24:56.270 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:24:56.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:24:56.273 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T14:24:56.578 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7623211533911619) > weight (0.25) 2021-02-20T14:24:56.578 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7586185784157211) > weight (0.25) 2021-02-20T14:24:56.579 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8885261857600928) > weight (0.25) 2021-02-20T14:24:56.579 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.26150534591481445) > weight (0.25) 2021-02-20T14:24:56.580 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T14:26:17.275 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.27760690280638944) > weight (0.25) 2021-02-20T14:26:17.303 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7685955098294688) > weight (0.25) 2021-02-20T14:26:17.304 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.39369979387279863) > weight (0.25) 2021-02-20T14:26:17.304 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T14:26:17.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T14:26:17.305 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T14:26:17.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T14:26:17.304+0000 7f3e19d5c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 60912) UID: 1000 2021-02-20T14:26:17.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:26:18.306 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T14:26:18.306 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:26:18.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:26:18.307 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T14:26:18.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T14:26:18.308 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T14:26:18.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T14:26:18.317 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T14:26:40.580 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.38595793265782563) > weight (0.25) 2021-02-20T14:26:40.588 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8382216343746681) > weight (0.25) 2021-02-20T14:26:40.588 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3319918384099442) > weight (0.25) 2021-02-20T14:26:40.589 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8864215151813676) > weight (0.25) 2021-02-20T14:26:40.589 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T14:27:57.318 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6729109844948218) > weight (0.25) 2021-02-20T14:27:57.337 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5969942764420533) > weight (0.25) 2021-02-20T14:27:57.337 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.967016555101228) > weight (0.25) 2021-02-20T14:27:57.338 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.44296687402172563) > weight (0.25) 2021-02-20T14:27:57.338 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T14:28:27.590 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8221092351079194) > weight (0.25) 2021-02-20T14:28:27.597 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:28:27.598 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:28:27.599 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T14:28:27.614 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T14:28:27.610+0000 7f2e8b083700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 60448) UID: 1000 2021-02-20T14:28:27.814 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:28:32.600 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:28:32.607 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:28:32.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:28:32.608 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:28:32.608 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:28:32.609 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:28:32.611 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:28:32.612 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T14:29:41.343 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.31006707089838714) > weight (0.25) 2021-02-20T14:29:41.350 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T14:29:41.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T14:29:41.352 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T14:29:41.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T14:29:41.349+0000 7f97f9824700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 60354) UID: 1000 2021-02-20T14:29:41.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:29:48.353 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T14:29:48.354 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:29:48.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:29:48.355 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T14:29:48.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T14:29:48.356 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T14:29:48.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T14:29:48.371 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T14:30:06.613 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:30:06.624 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:30:06.625 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T14:30:06.626 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T14:30:06.624+0000 7fc012a6f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 60722) UID: 1000 2021-02-20T14:30:06.826 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:30:09.626 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:30:09.627 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:30:09.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:30:09.628 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:30:09.628 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:30:09.628 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:30:09.631 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:30:09.632 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T14:31:23.633 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7310520231577196) > weight (0.25) 2021-02-20T14:31:23.643 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6217175217807612) > weight (0.25) 2021-02-20T14:31:23.643 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T14:31:23.644 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T14:31:23.644 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T14:31:23.645 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T14:31:23.643+0000 7f5f2134a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 60539) UID: 1000 2021-02-20T14:31:24.046 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:31:31.645 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T14:31:32.294 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:31:32.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:31:32.296 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T14:31:32.296 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T14:31:32.297 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T14:31:32.310 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T14:31:32.311 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T14:31:44.373 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9895810281283612) > weight (0.25) 2021-02-20T14:31:44.380 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T14:31:44.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T14:31:44.381 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T14:31:44.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T14:31:44.379+0000 7f13db209700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 61288) UID: 1000 2021-02-20T14:31:44.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:31:46.382 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T14:31:46.383 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:31:46.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:31:46.383 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T14:31:46.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T14:31:46.384 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T14:31:46.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T14:31:46.388 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T14:33:12.312 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.599782545304431) > weight (0.25) 2021-02-20T14:33:12.318 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:33:12.319 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:33:12.319 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T14:33:12.320 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T14:33:12.317+0000 7f92d1205700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 61189) UID: 1000 2021-02-20T14:33:12.721 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:33:16.321 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:33:16.321 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:33:16.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:33:16.322 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:33:16.322 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:33:16.323 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:33:16.336 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:33:16.337 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T14:33:45.389 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8696003115388841) > weight (0.25) 2021-02-20T14:33:45.390 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6438355904746355) > weight (0.25) 2021-02-20T14:33:45.390 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4138864285882421) > weight (0.25) 2021-02-20T14:33:45.391 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T14:33:45.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T14:33:45.392 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T14:33:45.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T14:33:45.392+0000 7f436df46700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 61098) UID: 1000 2021-02-20T14:33:45.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:33:52.392 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T14:33:52.397 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:33:52.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:33:52.398 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T14:33:52.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T14:33:52.399 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T14:33:52.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T14:33:52.403 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T14:35:09.338 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5088744663595834) > weight (0.25) 2021-02-20T14:35:09.349 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5367498732211097) > weight (0.25) 2021-02-20T14:35:09.349 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T14:35:09.350 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T14:35:09.356 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T14:35:09.358 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T14:35:09.347+0000 7f9b4cb24700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 61471) UID: 1000 2021-02-20T14:35:09.551 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:35:11.358 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T14:35:11.358 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:35:11.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:35:11.359 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T14:35:11.359 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T14:35:11.360 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T14:35:11.373 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T14:35:11.373 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T14:35:24.404 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5039681147076229) > weight (0.25) 2021-02-20T14:35:24.412 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8123790300946878) > weight (0.25) 2021-02-20T14:35:24.412 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3083013715370432) > weight (0.25) 2021-02-20T14:35:24.413 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6499716834914868) > weight (0.25) 2021-02-20T14:35:24.413 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T14:36:22.375 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5110530232237842) > weight (0.25) 2021-02-20T14:36:22.383 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:36:22.384 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:36:22.385 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T14:36:22.386 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T14:36:22.382+0000 7fb3fb79e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 61654) UID: 1000 2021-02-20T14:36:22.586 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:36:24.010 INFO:tasks.ceph.cluster2.osd.1.smithi090.stderr:2021-02-20T14:36:24.006+0000 7f38efdb0700 -1 reset not still connected to 0x5649845fe340 2021-02-20T14:36:24.010 INFO:tasks.ceph.cluster2.osd.1.smithi090.stderr:2021-02-20T14:36:24.006+0000 7f38efdb0700 -1 reset not still connected to 0x5649845ff040 2021-02-20T14:36:27.385 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:36:27.386 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:36:27.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:36:27.387 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:36:27.387 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:36:27.388 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:36:27.391 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:36:27.391 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T14:37:03.415 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5394949332722325) > weight (0.25) 2021-02-20T14:37:03.424 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T14:37:03.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T14:37:03.425 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T14:37:03.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T14:37:03.425+0000 7f98702b3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 61562) UID: 1000 2021-02-20T14:37:03.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:37:12.426 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T14:37:12.426 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:37:12.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:37:12.427 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T14:37:12.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T14:37:12.428 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T14:37:12.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T14:37:12.431 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T14:37:41.392 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.42013477218613204) > weight (0.25) 2021-02-20T14:37:41.393 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5400865987499448) > weight (0.25) 2021-02-20T14:37:41.393 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7771757996467639) > weight (0.25) 2021-02-20T14:37:41.394 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9666399149811914) > weight (0.25) 2021-02-20T14:37:41.395 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T14:38:50.432 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.25249371986637115) > weight (0.25) 2021-02-20T14:38:50.515 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8297250673529014) > weight (0.25) 2021-02-20T14:38:50.516 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8168323898453305) > weight (0.25) 2021-02-20T14:38:50.516 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9399336586524838) > weight (0.25) 2021-02-20T14:38:50.517 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T14:39:11.396 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6936533241408364) > weight (0.25) 2021-02-20T14:39:11.397 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:39:11.399 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:39:11.399 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T14:39:11.400 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T14:39:11.399+0000 7ff1446a4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 61930) UID: 1000 2021-02-20T14:39:11.600 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:39:15.403 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:39:15.410 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:39:15.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:39:15.411 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:39:15.411 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:39:15.412 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:39:15.414 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:39:15.415 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T14:40:35.518 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7862582538192905) > weight (0.25) 2021-02-20T14:40:35.520 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T14:40:35.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T14:40:35.522 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T14:40:35.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T14:40:35.522+0000 7f468e666700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 62021) UID: 1000 2021-02-20T14:40:35.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:40:42.523 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T14:40:42.535 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:40:42.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:40:42.536 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T14:40:42.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T14:40:42.537 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T14:40:42.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T14:40:42.540 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T14:40:58.415 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:40:58.423 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:40:58.424 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T14:40:58.425 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T14:40:58.421+0000 7fb2fb3fd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 61380) UID: 1000 2021-02-20T14:40:58.825 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:41:07.425 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:41:07.433 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:41:07.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:41:07.433 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:41:07.434 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:41:07.434 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:41:07.437 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:41:07.437 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T14:42:28.438 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:42:28.447 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:42:28.447 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T14:42:28.448 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T14:42:28.448+0000 7f85e3c43700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 62296) UID: 1000 2021-02-20T14:42:28.649 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:42:33.449 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:42:33.449 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:42:33.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:42:33.450 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:42:33.451 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:42:33.451 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:42:33.455 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:42:33.455 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T14:42:35.541 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6668769061671551) > weight (0.25) 2021-02-20T14:42:35.558 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6530526740089173) > weight (0.25) 2021-02-20T14:42:35.558 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.433511342185829) > weight (0.25) 2021-02-20T14:42:35.559 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.355308260176902) > weight (0.25) 2021-02-20T14:42:35.559 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T14:43:40.560 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9769793857678256) > weight (0.25) 2021-02-20T14:43:40.561 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4427265697691243) > weight (0.25) 2021-02-20T14:43:40.562 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4000667260499562) > weight (0.25) 2021-02-20T14:43:40.562 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.406548474622769) > weight (0.25) 2021-02-20T14:43:40.562 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T14:44:23.456 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:44:23.458 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:44:23.458 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T14:44:23.458 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:44:23.459 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:44:23.460 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T14:44:23.458+0000 7f1d50cb3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 62387) UID: 1000 2021-02-20T14:44:23.660 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:44:29.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:44:29.561 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:44:29.561 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:44:29.562 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:44:29.565 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:44:29.566 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T14:45:37.564 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.764131367961904) > weight (0.25) 2021-02-20T14:45:37.564 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9076764453669487) > weight (0.25) 2021-02-20T14:45:37.565 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T14:45:37.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T14:45:37.566 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T14:45:37.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T14:45:37.564+0000 7f04a14fd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 60813) UID: 1000 2021-02-20T14:45:37.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:45:42.567 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T14:45:42.568 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:45:42.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:45:42.569 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T14:45:42.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T14:45:42.569 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T14:45:42.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T14:45:42.573 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T14:45:57.243 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T14:45:57.240+0000 7f7308f93700 -1 reset not still connected to 0x55aaca8f7930 2021-02-20T14:46:19.567 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:46:19.568 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:46:19.569 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T14:46:19.569 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:46:19.569 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:46:19.570 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T14:46:19.568+0000 7f6a0fa47700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 62479) UID: 1000 2021-02-20T14:46:19.770 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:46:25.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:46:25.671 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:46:25.671 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:46:25.672 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:46:25.674 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:46:25.675 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T14:47:34.574 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6532789311790368) > weight (0.25) 2021-02-20T14:47:34.575 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.43453578366473966) > weight (0.25) 2021-02-20T14:47:34.575 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8084761536956918) > weight (0.25) 2021-02-20T14:47:34.575 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.68926318800161) > weight (0.25) 2021-02-20T14:47:34.576 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T14:48:24.676 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9289427738278982) > weight (0.25) 2021-02-20T14:48:24.680 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:48:24.681 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:48:24.681 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T14:48:24.683 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T14:48:24.682+0000 7f0d02f16700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 62112) UID: 1000 2021-02-20T14:48:24.883 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:48:25.682 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:48:25.683 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:48:25.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:48:25.684 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:48:25.684 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:48:25.685 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:48:25.689 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:48:25.689 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T14:48:34.576 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.670607684543957) > weight (0.25) 2021-02-20T14:48:34.589 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T14:48:34.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T14:48:34.591 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T14:48:34.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T14:48:34.589+0000 7f8299ab3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 62205) UID: 1000 2021-02-20T14:48:34.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:48:43.592 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T14:48:43.600 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:48:43.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:48:43.601 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T14:48:43.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T14:48:43.602 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T14:48:43.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T14:48:43.605 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T14:49:37.691 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:49:37.704 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:49:37.705 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T14:49:37.705 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:49:37.706 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:49:37.707 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T14:49:37.704+0000 7fbd79d80700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 62672) UID: 1000 2021-02-20T14:49:37.906 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:49:43.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:49:43.812 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:49:43.812 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:49:43.812 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:49:43.815 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:49:43.815 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T14:50:10.606 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5385601121118917) > weight (0.25) 2021-02-20T14:50:10.695 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5308833615053767) > weight (0.25) 2021-02-20T14:50:10.696 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3274691751407346) > weight (0.25) 2021-02-20T14:50:10.696 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9945955678465179) > weight (0.25) 2021-02-20T14:50:10.696 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T14:50:40.766 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T14:50:40.763+0000 7f7308f93700 -1 reset not still connected to 0x55aab67cc3c0 2021-02-20T14:50:40.767 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T14:50:40.763+0000 7f7308f93700 -1 reset not still connected to 0x55aab67cda80 2021-02-20T14:50:40.767 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T14:50:40.763+0000 7f7308f93700 -1 reset not still connected to 0x55aac23594d0 2021-02-20T14:51:00.817 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:51:00.825 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:51:00.825 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T14:51:00.826 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T14:51:00.823+0000 7fc05592d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 62946) UID: 1000 2021-02-20T14:51:01.027 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:51:02.826 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:51:02.827 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:51:02.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:51:02.828 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:51:02.829 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:51:02.829 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:51:02.835 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:51:02.835 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T14:51:57.698 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4803045697445979) > weight (0.25) 2021-02-20T14:51:57.699 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7245468087100855) > weight (0.25) 2021-02-20T14:51:57.699 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5673384398595058) > weight (0.25) 2021-02-20T14:51:57.699 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9960341432857325) > weight (0.25) 2021-02-20T14:51:57.699 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T14:52:29.836 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6836825929654373) > weight (0.25) 2021-02-20T14:52:29.837 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:52:29.838 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:52:29.838 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T14:52:29.839 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T14:52:29.837+0000 7efdae2b6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 62763) UID: 1000 2021-02-20T14:52:30.040 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:52:34.839 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:52:34.839 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:52:34.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:52:34.840 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:52:34.840 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:52:34.841 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:52:34.843 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:52:34.845 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T14:53:12.700 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7716861632989637) > weight (0.25) 2021-02-20T14:53:12.701 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5127251071782398) > weight (0.25) 2021-02-20T14:53:12.701 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.799013724642617) > weight (0.25) 2021-02-20T14:53:12.701 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9521616511528299) > weight (0.25) 2021-02-20T14:53:12.701 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T14:54:00.847 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:54:00.848 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:54:00.849 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T14:54:00.850 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T14:54:00.848+0000 7fd108a4b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 63038) UID: 1000 2021-02-20T14:54:01.050 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:54:08.850 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:54:08.850 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:54:08.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:54:08.851 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:54:08.852 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:54:08.852 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:54:08.855 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:54:08.856 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T14:55:05.703 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:55:05.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:55:05.704 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T14:55:05.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T14:55:05.703+0000 7fc655470700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 61004) UID: 1000 2021-02-20T14:55:05.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:55:10.705 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:55:10.706 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:55:10.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:55:10.706 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:55:10.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:55:10.707 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:55:10.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:55:10.710 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T14:55:45.856 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.910383097782126) > weight (0.25) 2021-02-20T14:55:45.857 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5795177888186316) > weight (0.25) 2021-02-20T14:55:45.857 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7065686091160003) > weight (0.25) 2021-02-20T14:55:45.858 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6607475528400686) > weight (0.25) 2021-02-20T14:55:45.858 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T14:56:21.711 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.941004449867937) > weight (0.25) 2021-02-20T14:56:21.712 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2675088808959477) > weight (0.25) 2021-02-20T14:56:21.712 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4446207009301738) > weight (0.25) 2021-02-20T14:56:21.713 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.48419041450583933) > weight (0.25) 2021-02-20T14:56:21.713 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T14:57:42.859 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6649943074706167) > weight (0.25) 2021-02-20T14:57:42.860 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:57:42.861 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:57:42.862 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T14:57:42.863 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T14:57:42.860+0000 7f680f84f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 63129) UID: 1000 2021-02-20T14:57:43.063 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:57:49.863 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:57:49.863 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:57:49.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:57:49.864 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:57:49.864 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:57:49.865 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:57:49.867 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:57:49.868 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T14:57:59.714 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:57:59.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:57:59.715 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T14:57:59.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T14:57:59.716+0000 7fcb47b14700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 63314) UID: 1000 2021-02-20T14:57:59.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:58:00.715 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:58:00.716 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:58:00.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:58:00.716 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:58:00.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:58:00.717 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:58:00.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:58:00.720 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T14:59:00.721 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.46454496296370607) > weight (0.25) 2021-02-20T14:59:00.730 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4941069936470519) > weight (0.25) 2021-02-20T14:59:00.730 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7424484922241933) > weight (0.25) 2021-02-20T14:59:00.730 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.788233791969206) > weight (0.25) 2021-02-20T14:59:00.730 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T14:59:15.869 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:59:15.976 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:59:15.977 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T14:59:15.977 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T14:59:15.974+0000 7f1b33323700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 63220) UID: 1000 2021-02-20T14:59:16.178 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:59:23.978 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:59:23.979 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:59:23.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:59:23.980 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:59:23.980 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:59:23.980 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:59:23.983 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:59:23.984 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T15:00:02.732 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6071603451755847) > weight (0.25) 2021-02-20T15:00:02.740 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.28662355496980263) > weight (0.25) 2021-02-20T15:00:02.740 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.49893107315850616) > weight (0.25) 2021-02-20T15:00:02.741 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6734804359233606) > weight (0.25) 2021-02-20T15:00:02.741 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T15:01:03.985 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4884360397270596) > weight (0.25) 2021-02-20T15:01:03.986 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.33525341243962514) > weight (0.25) 2021-02-20T15:01:03.986 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5981908928038268) > weight (0.25) 2021-02-20T15:01:03.986 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T15:01:03.987 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T15:01:03.988 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T15:01:03.989 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T15:01:03.985+0000 7f506690b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 60077) UID: 1000 2021-02-20T15:01:04.389 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:01:09.989 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T15:01:09.989 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:01:09.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:01:09.990 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T15:01:09.991 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T15:01:09.991 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T15:01:09.994 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T15:01:09.994 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T15:01:30.742 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:01:30.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:01:30.743 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T15:01:30.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T15:01:30.744+0000 7f2ad779f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 63497) UID: 1000 2021-02-20T15:01:30.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:01:37.745 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:01:37.745 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:01:37.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:01:37.746 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:01:37.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:01:37.747 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:01:37.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:01:37.750 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T15:02:32.995 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:02:33.004 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:02:33.004 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T15:02:33.005 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T15:02:33.003+0000 7fd401ef6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 63589) UID: 1000 2021-02-20T15:02:33.205 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:02:38.006 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:02:38.006 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:02:38.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:02:38.007 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:02:38.007 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:02:38.008 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:02:38.012 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:02:38.012 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T15:03:21.751 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5476183494881987) > weight (0.25) 2021-02-20T15:03:21.752 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T15:03:21.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T15:03:21.753 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T15:03:21.753 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T15:03:21.754 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:03:21.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T15:03:21.750+0000 7f7c8ae0c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 62854) UID: 1000 2021-02-20T15:03:21.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:03:27.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:03:27.857 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T15:03:27.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T15:03:27.857 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T15:03:27.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T15:03:27.860 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T15:03:44.013 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8908390807194893) > weight (0.25) 2021-02-20T15:03:44.014 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.37356547865235856) > weight (0.25) 2021-02-20T15:03:44.014 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8973559312260065) > weight (0.25) 2021-02-20T15:03:44.014 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T15:03:44.015 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T15:03:44.015 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:03:44.017 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T15:03:44.014+0000 7fef9c830700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 63681) UID: 1000 2021-02-20T15:03:44.218 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:03:52.016 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T15:03:52.017 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:03:52.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:03:52.017 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T15:03:52.018 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T15:03:52.018 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T15:03:52.020 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T15:03:52.021 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T15:04:31.861 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6630538768977534) > weight (0.25) 2021-02-20T15:04:31.862 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4000910377812089) > weight (0.25) 2021-02-20T15:04:31.862 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.46801472523210264) > weight (0.25) 2021-02-20T15:04:31.862 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.36700546873398554) > weight (0.25) 2021-02-20T15:04:31.863 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T15:05:21.022 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3679967469363863) > weight (0.25) 2021-02-20T15:05:21.023 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7592468713801608) > weight (0.25) 2021-02-20T15:05:21.023 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T15:05:21.024 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T15:05:21.024 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T15:05:21.025 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T15:05:21.024+0000 7f62b609f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 61839) UID: 1000 2021-02-20T15:05:21.226 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:05:26.026 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T15:05:26.026 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:05:26.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:05:26.027 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T15:05:26.027 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T15:05:26.028 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T15:05:26.031 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T15:05:26.031 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T15:05:31.864 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:05:31.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:05:31.865 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T15:05:31.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T15:05:31.864+0000 7f00231a3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 63772) UID: 1000 2021-02-20T15:05:32.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:05:33.866 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:05:33.866 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:05:33.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:05:33.867 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:05:33.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:05:33.868 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:05:33.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:05:33.871 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T15:07:18.032 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.533647481263684) > weight (0.25) 2021-02-20T15:07:18.033 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6991014002225872) > weight (0.25) 2021-02-20T15:07:18.033 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T15:07:18.034 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T15:07:18.035 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T15:07:18.036 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T15:07:18.030+0000 7fdda2f17700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 64141) UID: 1000 2021-02-20T15:07:18.236 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:07:24.036 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T15:07:24.036 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:07:24.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:07:24.038 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T15:07:24.038 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T15:07:24.039 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T15:07:24.042 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T15:07:24.043 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T15:07:24.872 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4825159004846843) > weight (0.25) 2021-02-20T15:07:24.873 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5486471151351874) > weight (0.25) 2021-02-20T15:07:24.873 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6804354511202892) > weight (0.25) 2021-02-20T15:07:24.873 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.40607399499073227) > weight (0.25) 2021-02-20T15:07:24.873 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T15:08:30.874 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.571217494332817) > weight (0.25) 2021-02-20T15:08:30.882 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5555840275890094) > weight (0.25) 2021-02-20T15:08:30.883 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5724333352839076) > weight (0.25) 2021-02-20T15:08:30.883 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.42679122406374215) > weight (0.25) 2021-02-20T15:08:30.883 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T15:08:35.044 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:08:35.054 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:08:35.054 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T15:08:35.055 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T15:08:35.053+0000 7fadabe85700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 63863) UID: 1000 2021-02-20T15:08:35.256 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:08:42.056 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:08:42.067 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:08:42.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:08:42.068 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:08:42.068 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:08:42.068 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:08:42.071 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:08:42.072 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T15:09:41.884 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9560305291258498) > weight (0.25) 2021-02-20T15:09:41.889 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9149316070796982) > weight (0.25) 2021-02-20T15:09:41.889 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2665554325119617) > weight (0.25) 2021-02-20T15:09:41.889 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T15:09:41.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T15:09:41.890 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T15:09:41.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T15:09:41.888+0000 7f0a5afde700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 61745) UID: 1000 2021-02-20T15:09:42.183 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T15:09:42.180+0000 7f7308f93700 -1 reset not still connected to 0x55aac40b3e90 2021-02-20T15:09:42.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:09:50.891 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T15:09:50.898 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:09:50.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:09:50.899 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T15:09:50.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T15:09:50.900 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T15:09:50.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T15:09:50.903 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T15:10:04.073 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5411477304191359) > weight (0.25) 2021-02-20T15:10:04.084 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.29186823174617094) > weight (0.25) 2021-02-20T15:10:04.084 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8818197254098881) > weight (0.25) 2021-02-20T15:10:04.085 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8101082598888952) > weight (0.25) 2021-02-20T15:10:04.085 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T15:11:39.904 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4643243931816393) > weight (0.25) 2021-02-20T15:11:39.915 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9597375672382988) > weight (0.25) 2021-02-20T15:11:39.915 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.528756995490756) > weight (0.25) 2021-02-20T15:11:39.915 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9094983934946245) > weight (0.25) 2021-02-20T15:11:39.916 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T15:11:56.086 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:11:56.094 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:11:56.095 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T15:11:56.096 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T15:11:56.090+0000 7ffadc33c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 64415) UID: 1000 2021-02-20T15:11:56.296 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:12:03.096 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:12:03.104 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:12:03.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:12:03.105 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:12:03.105 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:12:03.106 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:12:03.109 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:12:03.109 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T15:13:23.917 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5790297372901952) > weight (0.25) 2021-02-20T15:13:23.918 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6110568912376338) > weight (0.25) 2021-02-20T15:13:23.918 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7069017503448835) > weight (0.25) 2021-02-20T15:13:23.918 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6427653593466988) > weight (0.25) 2021-02-20T15:13:23.919 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T15:13:59.110 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7697836522170691) > weight (0.25) 2021-02-20T15:13:59.111 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.32703156712414794) > weight (0.25) 2021-02-20T15:13:59.111 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7963882304136769) > weight (0.25) 2021-02-20T15:13:59.111 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7521139191739433) > weight (0.25) 2021-02-20T15:13:59.111 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T15:14:28.920 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.37358811102748435) > weight (0.25) 2021-02-20T15:14:28.920 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9754876583141118) > weight (0.25) 2021-02-20T15:14:28.921 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T15:14:28.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T15:14:28.922 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T15:14:28.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T15:14:28.919+0000 7fcf41af2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 62581) UID: 1000 2021-02-20T15:14:29.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:14:35.924 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T15:14:35.924 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:14:35.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:14:35.926 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T15:14:35.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T15:14:35.927 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T15:14:35.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T15:14:35.932 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T15:15:32.113 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7128253687448639) > weight (0.25) 2021-02-20T15:15:32.114 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6891505541913071) > weight (0.25) 2021-02-20T15:15:32.114 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7499154994584766) > weight (0.25) 2021-02-20T15:15:32.114 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2595846200892631) > weight (0.25) 2021-02-20T15:15:32.114 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T15:15:42.933 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:15:42.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:15:42.935 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T15:15:42.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T15:15:42.930+0000 7f18de43f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 64232) UID: 1000 2021-02-20T15:15:43.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:15:51.935 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:15:51.936 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:15:51.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:15:51.937 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:15:51.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:15:51.938 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:15:51.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:15:51.941 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T15:16:44.115 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7977037107780285) > weight (0.25) 2021-02-20T15:16:44.116 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.35143505909458317) > weight (0.25) 2021-02-20T15:16:44.116 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5383589874470994) > weight (0.25) 2021-02-20T15:16:44.117 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2893166112104466) > weight (0.25) 2021-02-20T15:16:44.117 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T15:16:56.402 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T15:16:56.392+0000 7f7308f93700 -1 reset not still connected to 0x55aac0012b10 2021-02-20T15:16:56.402 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T15:16:56.396+0000 7f7308f93700 -1 reset not still connected to 0x55aac3f847d0 2021-02-20T15:16:56.403 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T15:16:56.396+0000 7f7308f93700 -1 reset not still connected to 0x55aac40b2e50 2021-02-20T15:16:56.403 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T15:16:56.396+0000 7f7308f93700 -1 reset not still connected to 0x55aacd8bef70 2021-02-20T15:17:35.942 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.707204489766099) > weight (0.25) 2021-02-20T15:17:35.943 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.716178898374087) > weight (0.25) 2021-02-20T15:17:35.943 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4618363086425653) > weight (0.25) 2021-02-20T15:17:35.943 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8799402122552459) > weight (0.25) 2021-02-20T15:17:35.943 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T15:17:59.117 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8296026227969351) > weight (0.25) 2021-02-20T15:17:59.118 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8211725917383673) > weight (0.25) 2021-02-20T15:17:59.118 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5000359941751541) > weight (0.25) 2021-02-20T15:17:59.119 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2760136696003458) > weight (0.25) 2021-02-20T15:17:59.119 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T15:19:19.945 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6888793651232475) > weight (0.25) 2021-02-20T15:19:19.946 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8473619972368663) > weight (0.25) 2021-02-20T15:19:19.946 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7775520687540869) > weight (0.25) 2021-02-20T15:19:19.946 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4142216668061063) > weight (0.25) 2021-02-20T15:19:19.947 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T15:19:21.120 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8181278846178027) > weight (0.25) 2021-02-20T15:19:21.120 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6119122951833204) > weight (0.25) 2021-02-20T15:19:21.120 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.33100640702654927) > weight (0.25) 2021-02-20T15:19:21.121 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2847884478697942) > weight (0.25) 2021-02-20T15:19:21.121 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T15:21:01.948 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:21:01.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:21:01.950 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T15:21:01.950 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:21:01.950 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:21:01.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T15:21:01.944+0000 7f3e6c75f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 64800) UID: 1000 2021-02-20T15:21:02.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:21:06.122 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2752093517448493) > weight (0.25) 2021-02-20T15:21:06.123 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8480971185505182) > weight (0.25) 2021-02-20T15:21:06.123 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.28442283049318995) > weight (0.25) 2021-02-20T15:21:06.123 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5898444834629482) > weight (0.25) 2021-02-20T15:21:06.123 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T15:21:08.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:21:08.053 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:21:08.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:21:08.054 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:21:08.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:21:08.058 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T15:22:24.125 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8486535046304453) > weight (0.25) 2021-02-20T15:22:24.130 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T15:22:24.131 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T15:22:24.131 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T15:22:24.132 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T15:22:24.131+0000 7efe54a5e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 63406) UID: 1000 2021-02-20T15:22:24.533 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:22:28.132 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T15:22:28.133 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:22:28.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:22:28.134 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T15:22:28.134 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T15:22:28.134 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T15:22:28.137 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T15:22:28.137 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T15:22:52.060 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4822771573501683) > weight (0.25) 2021-02-20T15:22:52.066 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.26113287477810043) > weight (0.25) 2021-02-20T15:22:52.067 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8326458985950396) > weight (0.25) 2021-02-20T15:22:52.067 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7401480608276796) > weight (0.25) 2021-02-20T15:22:52.067 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T15:23:57.069 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8529463383354461) > weight (0.25) 2021-02-20T15:23:57.069 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5165722221963598) > weight (0.25) 2021-02-20T15:23:57.070 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6338049204115284) > weight (0.25) 2021-02-20T15:23:57.070 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6206672449124531) > weight (0.25) 2021-02-20T15:23:57.070 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T15:23:57.138 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4792322452831166) > weight (0.25) 2021-02-20T15:23:57.138 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4600563462250543) > weight (0.25) 2021-02-20T15:23:57.139 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5351804592890577) > weight (0.25) 2021-02-20T15:23:57.139 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7000546530219274) > weight (0.25) 2021-02-20T15:23:57.139 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T15:25:14.071 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7948421488207039) > weight (0.25) 2021-02-20T15:25:14.072 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9491092587973695) > weight (0.25) 2021-02-20T15:25:14.072 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T15:25:14.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T15:25:14.073 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T15:25:14.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T15:25:14.072+0000 7fe947bbd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 64698) UID: 1000 2021-02-20T15:25:14.140 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:25:14.141 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:25:14.142 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T15:25:14.142 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T15:25:14.140+0000 7f64964e9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 64606) UID: 1000 2021-02-20T15:25:14.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:25:14.343 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:25:18.143 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:25:18.143 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:25:18.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:25:18.144 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:25:18.144 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:25:18.145 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:25:18.147 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:25:18.148 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T15:25:23.075 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T15:25:23.075 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:25:23.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:25:23.076 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T15:25:23.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T15:25:23.076 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T15:25:23.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T15:25:23.080 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T15:26:21.149 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2642025476244818) > weight (0.25) 2021-02-20T15:26:21.150 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6816544728555729) > weight (0.25) 2021-02-20T15:26:21.150 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T15:26:21.151 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T15:26:21.152 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T15:26:21.153 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T15:26:21.151+0000 7f78b2623700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 64324) UID: 1000 2021-02-20T15:26:21.315 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T15:26:21.311+0000 7f7308f93700 -1 reset not still connected to 0x55aab68723c0 2021-02-20T15:26:21.316 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T15:26:21.311+0000 7f7308f93700 -1 reset not still connected to 0x55aacd8bf2b0 2021-02-20T15:26:21.553 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:26:28.153 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T15:26:28.153 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:26:28.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:26:28.154 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T15:26:28.155 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T15:26:28.155 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T15:26:28.158 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T15:26:28.158 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T15:27:21.081 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.549971707026923) > weight (0.25) 2021-02-20T15:27:21.081 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T15:27:21.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T15:27:21.082 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T15:27:21.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T15:27:21.078+0000 7f576e64f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 63954) UID: 1000 2021-02-20T15:27:21.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:27:25.084 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T15:27:25.084 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:27:25.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:27:25.085 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T15:27:25.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T15:27:25.086 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T15:27:25.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T15:27:25.089 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T15:28:19.160 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3472784367852606) > weight (0.25) 2021-02-20T15:28:19.160 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4517930130673532) > weight (0.25) 2021-02-20T15:28:19.161 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T15:28:19.161 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T15:28:19.162 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T15:28:19.162 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T15:28:19.162 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:28:19.163 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T15:28:19.157+0000 7f4624668700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 65264) UID: 1000 2021-02-20T15:28:19.363 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:28:25.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:28:25.265 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T15:28:25.265 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T15:28:25.265 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T15:28:25.268 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T15:28:25.269 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T15:28:50.089 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8955891723933413) > weight (0.25) 2021-02-20T15:28:50.090 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7524201730927239) > weight (0.25) 2021-02-20T15:28:50.090 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6507872154677532) > weight (0.25) 2021-02-20T15:28:50.091 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7006298861694135) > weight (0.25) 2021-02-20T15:28:50.091 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T15:30:05.270 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.32739080763133155) > weight (0.25) 2021-02-20T15:30:05.271 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3746415627380628) > weight (0.25) 2021-02-20T15:30:05.271 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5229164032124413) > weight (0.25) 2021-02-20T15:30:05.271 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.624029598156961) > weight (0.25) 2021-02-20T15:30:05.271 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T15:30:24.092 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6486410991921355) > weight (0.25) 2021-02-20T15:30:24.093 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8640457548595705) > weight (0.25) 2021-02-20T15:30:24.093 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.687001449975402) > weight (0.25) 2021-02-20T15:30:24.093 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8467031681853334) > weight (0.25) 2021-02-20T15:30:24.094 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T15:31:16.273 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3510835870720074) > weight (0.25) 2021-02-20T15:31:16.274 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.49782675070347193) > weight (0.25) 2021-02-20T15:31:16.274 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.48754465989157547) > weight (0.25) 2021-02-20T15:31:16.274 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4023555852744706) > weight (0.25) 2021-02-20T15:31:16.274 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T15:31:50.095 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6738792026916006) > weight (0.25) 2021-02-20T15:31:50.096 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T15:31:50.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T15:31:50.097 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T15:31:50.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T15:31:50.093+0000 7f1343b57700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 65355) UID: 1000 2021-02-20T15:31:50.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:31:53.099 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T15:31:53.099 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:31:53.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:31:53.100 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T15:31:53.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T15:31:53.101 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T15:31:53.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T15:31:53.104 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T15:33:05.275 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.36825408500073464) > weight (0.25) 2021-02-20T15:33:05.276 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2899555838353316) > weight (0.25) 2021-02-20T15:33:05.276 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3239378631413362) > weight (0.25) 2021-02-20T15:33:05.276 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T15:33:05.277 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T15:33:05.278 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T15:33:05.279 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T15:33:05.276+0000 7f2d257d4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 64046) UID: 1000 2021-02-20T15:33:05.479 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:33:14.279 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T15:33:14.279 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:33:14.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:33:14.280 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T15:33:14.280 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T15:33:14.281 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T15:33:14.284 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T15:33:14.284 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T15:33:40.106 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9694606770194788) > weight (0.25) 2021-02-20T15:33:40.106 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T15:33:40.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T15:33:40.107 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T15:33:40.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T15:33:40.103+0000 7f68b4535700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 65538) UID: 1000 2021-02-20T15:33:40.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:33:44.108 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T15:33:44.108 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:33:44.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:33:44.109 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T15:33:44.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T15:33:44.109 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T15:33:44.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T15:33:44.112 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T15:35:11.113 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5486162983091438) > weight (0.25) 2021-02-20T15:35:11.114 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.49847667375796134) > weight (0.25) 2021-02-20T15:35:11.114 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T15:35:11.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T15:35:11.115 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T15:35:11.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T15:35:11.114+0000 7f5f5d3dd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 65173) UID: 1000 2021-02-20T15:35:11.285 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:35:11.285 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:35:11.286 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T15:35:11.286 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:35:11.286 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:35:11.287 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T15:35:11.286+0000 7f9d3f9ee700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 65082) UID: 1000 2021-02-20T15:35:11.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:35:11.687 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:35:17.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:35:17.390 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:35:17.390 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:35:17.391 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:35:17.394 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:35:17.394 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T15:35:20.116 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T15:35:20.117 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:35:20.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:35:20.118 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T15:35:20.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T15:35:20.118 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T15:35:20.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T15:35:20.122 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T15:36:48.395 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7023766080093288) > weight (0.25) 2021-02-20T15:36:48.395 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5241468471656762) > weight (0.25) 2021-02-20T15:36:48.396 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T15:36:48.397 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T15:36:48.397 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T15:36:48.398 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T15:36:48.396+0000 7fd22da85700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 65446) UID: 1000 2021-02-20T15:36:48.598 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:36:53.397 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T15:36:53.398 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:36:53.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:36:53.399 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T15:36:53.399 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T15:36:53.399 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T15:36:53.402 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T15:36:53.402 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T15:37:16.123 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.893087030738486) > weight (0.25) 2021-02-20T15:37:16.124 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.43879531606792255) > weight (0.25) 2021-02-20T15:37:16.124 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4436031216250579) > weight (0.25) 2021-02-20T15:37:16.124 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T15:37:16.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T15:37:16.125 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T15:37:16.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T15:37:16.124+0000 7f8078be8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 64514) UID: 1000 2021-02-20T15:37:16.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:37:17.126 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T15:37:17.127 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:37:17.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:37:17.128 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T15:37:17.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T15:37:17.128 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T15:37:17.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T15:37:17.131 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T15:38:42.404 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.46765934623480887) > weight (0.25) 2021-02-20T15:38:42.405 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4553296269697944) > weight (0.25) 2021-02-20T15:38:42.405 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.54525904337981) > weight (0.25) 2021-02-20T15:38:42.406 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8750215560782805) > weight (0.25) 2021-02-20T15:38:42.406 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T15:39:09.133 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8166318737953733) > weight (0.25) 2021-02-20T15:39:09.133 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8156365803965261) > weight (0.25) 2021-02-20T15:39:09.134 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T15:39:09.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T15:39:09.135 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:39:09.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T15:39:09.134+0000 7f9b1cd53700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 65905) UID: 1000 2021-02-20T15:39:09.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:39:13.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T15:39:13.414+0000 7f97851af700 -1 rbd::mirror::ImageReplayer: 0x55560ad4db80 [3/3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd] start: already running 2021-02-20T15:39:17.136 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T15:39:17.137 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:39:17.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:39:17.138 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T15:39:17.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T15:39:17.138 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T15:39:17.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T15:39:17.141 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T15:39:17.370 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T15:39:17.366+0000 7fb533c45700 -1 reset not still connected to 0x558b8e4ddcf0 2021-02-20T15:39:17.371 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T15:39:17.366+0000 7fb533c45700 -1 reset not still connected to 0x558b8edd75f0 2021-02-20T15:39:17.371 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T15:39:17.366+0000 7fb533c45700 -1 reset not still connected to 0x558b8edd7860 2021-02-20T15:39:17.372 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T15:39:17.366+0000 7fb533c45700 -1 reset not still connected to 0x558b8edd83c0 2021-02-20T15:39:17.372 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T15:39:17.366+0000 7fb533c45700 -1 reset not still connected to 0x558b8edd9c20 2021-02-20T15:39:17.382 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T15:39:17.378+0000 7fb533c45700 -1 reset not still connected to 0x558b94650000 2021-02-20T15:39:17.382 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T15:39:17.378+0000 7fb533c45700 -1 reset not still connected to 0x558b94653c20 2021-02-20T15:39:17.382 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T15:39:17.378+0000 7fb533c45700 -1 reset not still connected to 0x558b94b8eea0 2021-02-20T15:39:17.383 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T15:39:17.378+0000 7fb533c45700 -1 reset not still connected to 0x558b95ecabe0 2021-02-20T15:39:17.383 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T15:39:17.378+0000 7fb533c45700 -1 reset not still connected to 0x558b9a1269c0 2021-02-20T15:39:17.383 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T15:39:17.378+0000 7fb533c45700 -1 reset not still connected to 0x558b9a126f70 2021-02-20T15:39:17.383 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T15:39:17.378+0000 7fb533c45700 -1 reset not still connected to 0x558b9a127860 2021-02-20T15:40:14.568 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T15:40:14.566+0000 7fb533c45700 -1 reset not still connected to 0x558b95d3f670 2021-02-20T15:40:14.569 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T15:40:14.566+0000 7fb533c45700 -1 reset not still connected to 0x558b95ecbb50 2021-02-20T15:40:24.407 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3364171365289469) > weight (0.25) 2021-02-20T15:40:24.408 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8523646369442756) > weight (0.25) 2021-02-20T15:40:24.408 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8482090739177245) > weight (0.25) 2021-02-20T15:40:24.408 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.802808098843061) > weight (0.25) 2021-02-20T15:40:24.409 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T15:40:43.143 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5272188444854715) > weight (0.25) 2021-02-20T15:40:43.143 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5364060218026787) > weight (0.25) 2021-02-20T15:40:43.144 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T15:40:43.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T15:40:43.144 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T15:40:43.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T15:40:43.144+0000 7fade52dc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 66195) UID: 1000 2021-02-20T15:40:43.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:40:50.145 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T15:40:50.145 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:40:50.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:40:50.147 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T15:40:50.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T15:40:50.147 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T15:40:50.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T15:40:50.151 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T15:42:06.410 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8700182234643012) > weight (0.25) 2021-02-20T15:42:06.411 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T15:42:06.412 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T15:42:06.412 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T15:42:06.413 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T15:42:06.411+0000 7f5eea6ea700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 64987) UID: 1000 2021-02-20T15:42:06.613 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:42:08.151 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:42:08.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:42:08.152 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:42:08.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T15:42:08.155+0000 7f247369d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 64896) UID: 1000 2021-02-20T15:42:08.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:42:12.413 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T15:42:12.414 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:42:12.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:42:12.415 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T15:42:12.416 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T15:42:12.416 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T15:42:12.420 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T15:42:12.420 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T15:42:16.153 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:42:16.153 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:42:16.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:42:16.154 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:42:16.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:42:16.155 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:42:16.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:42:16.158 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T15:44:05.158 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2899655913186189) > weight (0.25) 2021-02-20T15:44:05.159 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9756274946227067) > weight (0.25) 2021-02-20T15:44:05.159 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T15:44:05.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T15:44:05.161 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T15:44:05.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T15:44:05.157+0000 7f7d75cbf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 66287) UID: 1000 2021-02-20T15:44:05.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:44:08.162 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T15:44:08.162 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:44:08.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:44:08.163 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T15:44:08.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T15:44:08.164 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T15:44:08.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T15:44:08.167 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T15:44:10.421 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6460376913073994) > weight (0.25) 2021-02-20T15:44:10.422 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5402812572316745) > weight (0.25) 2021-02-20T15:44:10.422 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8690671397362172) > weight (0.25) 2021-02-20T15:44:10.422 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3085951093873045) > weight (0.25) 2021-02-20T15:44:10.423 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T15:45:09.168 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.579842476485025) > weight (0.25) 2021-02-20T15:45:09.169 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6523268808713134) > weight (0.25) 2021-02-20T15:45:09.169 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T15:45:09.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T15:45:09.170 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T15:45:09.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T15:45:09.167+0000 7ff538f34700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 66627) UID: 1000 2021-02-20T15:45:09.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:45:15.171 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T15:45:15.171 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:45:15.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:45:15.172 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T15:45:15.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T15:45:15.173 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T15:45:15.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T15:45:15.176 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T15:45:15.423 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7567112943145751) > weight (0.25) 2021-02-20T15:45:15.424 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T15:45:15.424 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T15:45:15.425 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T15:45:15.425 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T15:45:15.423+0000 7f31756eb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 66378) UID: 1000 2021-02-20T15:45:15.626 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:45:17.426 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T15:45:17.426 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:45:17.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:45:17.427 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T15:45:17.427 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T15:45:17.427 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T15:45:17.430 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T15:45:17.430 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T15:46:27.177 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2912893762847202) > weight (0.25) 2021-02-20T15:46:27.183 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T15:46:27.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T15:46:27.184 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T15:46:27.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T15:46:27.182+0000 7f16d0d67700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 65720) UID: 1000 2021-02-20T15:46:27.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:46:29.185 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T15:46:29.185 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:46:29.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:46:29.186 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T15:46:29.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T15:46:29.187 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T15:46:29.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T15:46:29.190 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T15:46:44.432 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:46:44.432 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:46:44.433 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T15:46:44.433 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:46:44.433 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:46:44.434 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T15:46:44.430+0000 7f142cd8c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 65814) UID: 1000 2021-02-20T15:46:44.834 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:46:50.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:46:50.535 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:46:50.535 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:46:50.536 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:46:50.538 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:46:50.539 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T15:47:35.191 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3481101213085208) > weight (0.25) 2021-02-20T15:47:35.192 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6418288189590793) > weight (0.25) 2021-02-20T15:47:35.192 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8936793021934193) > weight (0.25) 2021-02-20T15:47:35.192 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7310243786151946) > weight (0.25) 2021-02-20T15:47:35.193 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T15:47:51.540 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.592347751139567) > weight (0.25) 2021-02-20T15:47:51.541 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.29891920707455766) > weight (0.25) 2021-02-20T15:47:51.541 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T15:47:51.542 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T15:47:51.542 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T15:47:51.543 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T15:47:51.537+0000 7fe10b282700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 65997) UID: 1000 2021-02-20T15:47:51.944 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:47:56.543 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T15:47:56.544 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:47:56.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:47:56.544 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T15:47:56.545 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T15:47:56.546 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T15:47:56.550 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T15:47:56.550 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T15:49:07.552 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:49:07.553 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:49:07.554 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T15:49:07.555 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T15:49:07.551+0000 7fa0b8645700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 66996) UID: 1000 2021-02-20T15:49:07.755 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:49:09.555 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:49:09.555 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:49:09.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:49:09.556 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:49:09.556 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:49:09.557 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:49:09.559 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:49:09.560 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T15:49:25.194 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6200764246480843) > weight (0.25) 2021-02-20T15:49:25.195 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.302357176603937) > weight (0.25) 2021-02-20T15:49:25.195 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9774555961013257) > weight (0.25) 2021-02-20T15:49:25.196 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7731486400430536) > weight (0.25) 2021-02-20T15:49:25.196 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T15:50:51.561 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8950382110947966) > weight (0.25) 2021-02-20T15:50:51.562 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9468511537657066) > weight (0.25) 2021-02-20T15:50:51.562 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T15:50:51.563 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T15:50:51.563 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T15:50:51.564 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T15:50:51.558+0000 7f9bebc3e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 67088) UID: 1000 2021-02-20T15:50:51.764 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:50:57.564 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T15:50:57.564 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:50:57.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:50:57.565 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T15:50:57.565 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T15:50:57.566 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T15:50:57.569 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T15:50:57.569 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T15:51:05.197 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7023065753997721) > weight (0.25) 2021-02-20T15:51:05.197 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9784551044149872) > weight (0.25) 2021-02-20T15:51:05.198 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T15:51:05.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T15:51:05.199 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T15:51:05.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T15:51:05.193+0000 7f8b12d13700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 66723) UID: 1000 2021-02-20T15:51:05.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:51:06.200 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T15:51:06.200 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:51:06.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:51:06.201 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T15:51:06.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T15:51:06.202 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T15:51:06.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T15:51:06.205 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T15:52:21.571 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9401567289468142) > weight (0.25) 2021-02-20T15:52:21.572 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5765970551947125) > weight (0.25) 2021-02-20T15:52:21.572 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.33827645410664575) > weight (0.25) 2021-02-20T15:52:21.573 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6178410104404807) > weight (0.25) 2021-02-20T15:52:21.573 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T15:52:37.206 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.32351318980374744) > weight (0.25) 2021-02-20T15:52:37.206 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5186961889590788) > weight (0.25) 2021-02-20T15:52:37.207 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.888572672551767) > weight (0.25) 2021-02-20T15:52:37.207 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5421446924074182) > weight (0.25) 2021-02-20T15:52:37.207 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T15:52:44.565 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T15:52:44.560+0000 7f7308f93700 -1 reset not still connected to 0x55aac40b09c0 2021-02-20T15:52:44.566 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T15:52:44.560+0000 7f7308f93700 -1 reset not still connected to 0x55aaca8f83c0 2021-02-20T15:52:44.567 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T15:52:44.560+0000 7f7308f93700 -1 reset not still connected to 0x55aacd8bfa00 2021-02-20T15:54:01.575 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3894826331395921) > weight (0.25) 2021-02-20T15:54:01.576 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T15:54:01.576 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T15:54:01.577 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T15:54:01.578 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T15:54:01.574+0000 7f6dae9a2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 66814) UID: 1000 2021-02-20T15:54:01.979 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:54:05.578 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T15:54:05.579 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:54:05.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:54:05.580 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T15:54:05.580 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T15:54:05.581 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T15:54:05.584 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T15:54:05.584 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T15:54:10.207 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3514851262604759) > weight (0.25) 2021-02-20T15:54:10.208 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T15:54:10.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T15:54:10.209 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T15:54:10.209 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T15:54:10.209 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:54:10.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T15:54:10.206+0000 7fa2393f1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 66905) UID: 1000 2021-02-20T15:54:10.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:54:16.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:54:16.311 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T15:54:16.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T15:54:16.312 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T15:54:16.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T15:54:16.315 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T15:55:20.317 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7540114099887588) > weight (0.25) 2021-02-20T15:55:20.318 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.40765984256806387) > weight (0.25) 2021-02-20T15:55:20.318 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8850777501438765) > weight (0.25) 2021-02-20T15:55:20.318 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8477754235326089) > weight (0.25) 2021-02-20T15:55:20.318 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T15:55:36.586 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:55:36.587 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:55:36.588 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T15:55:36.589 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T15:55:36.585+0000 7ff3d3c15700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 67179) UID: 1000 2021-02-20T15:55:36.789 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:55:41.588 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:55:41.589 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:55:41.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:55:41.590 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:55:41.590 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:55:41.590 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:55:41.593 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:55:41.593 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T15:57:08.595 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:57:08.596 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:57:08.596 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T15:57:08.597 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T15:57:08.595+0000 7f9731c5c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 67638) UID: 1000 2021-02-20T15:57:08.798 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:57:14.598 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:57:14.602 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:57:14.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:57:14.603 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:57:14.603 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:57:14.603 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:57:14.606 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:57:14.607 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T15:57:19.320 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5583519687248598) > weight (0.25) 2021-02-20T15:57:19.320 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.38008874702850304) > weight (0.25) 2021-02-20T15:57:19.321 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9122347080202079) > weight (0.25) 2021-02-20T15:57:19.321 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6068456429860679) > weight (0.25) 2021-02-20T15:57:19.321 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T15:57:37.938 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T15:57:37.931+0000 7f7308f93700 -1 reset not still connected to 0x55aacd8c1cf0 2021-02-20T15:58:42.323 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8440357614799505) > weight (0.25) 2021-02-20T15:58:42.323 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.37124093061632146) > weight (0.25) 2021-02-20T15:58:42.324 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9688536642293417) > weight (0.25) 2021-02-20T15:58:42.324 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6309100740584069) > weight (0.25) 2021-02-20T15:58:42.324 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T15:59:10.608 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:59:10.609 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:59:10.610 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T15:59:10.611 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T15:59:10.605+0000 7f9246435700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 67729) UID: 1000 2021-02-20T15:59:10.811 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:59:15.611 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:59:15.611 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:59:15.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:59:15.612 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:59:15.612 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:59:15.613 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:59:15.616 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:59:15.616 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T16:00:03.325 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8840450324244292) > weight (0.25) 2021-02-20T16:00:03.325 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:00:03.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:00:03.326 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T16:00:03.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T16:00:03.320+0000 7f53f3eb5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 67544) UID: 1000 2021-02-20T16:00:03.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:00:05.327 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:00:05.328 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:00:05.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:00:05.329 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:00:05.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:00:05.329 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:00:05.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:00:05.332 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T16:00:41.617 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.38121792245815367) > weight (0.25) 2021-02-20T16:00:41.618 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:00:41.619 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:00:41.619 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T16:00:41.620 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T16:00:41.616+0000 7f81be84b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 67453) UID: 1000 2021-02-20T16:00:41.821 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:00:49.621 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:00:49.621 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:00:49.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:00:49.622 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:00:49.622 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:00:49.623 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:00:49.626 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:00:49.626 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T16:01:11.334 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6793764094983276) > weight (0.25) 2021-02-20T16:01:11.334 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:01:11.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:01:11.335 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T16:01:11.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T16:01:11.331+0000 7f1521460700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 67912) UID: 1000 2021-02-20T16:01:11.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:01:19.336 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:01:19.337 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:01:19.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:01:19.338 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:01:19.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:01:19.338 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:01:19.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:01:19.341 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T16:01:33.439 INFO:tasks.ceph.cluster2.osd.1.smithi090.stderr:2021-02-20T16:01:33.435+0000 7f38efdb0700 -1 reset not still connected to 0x56496e9d0270 2021-02-20T16:02:23.627 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9414676151959013) > weight (0.25) 2021-02-20T16:02:23.627 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5672325163015568) > weight (0.25) 2021-02-20T16:02:23.628 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6974156642187435) > weight (0.25) 2021-02-20T16:02:23.628 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8206365923393086) > weight (0.25) 2021-02-20T16:02:23.628 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T16:03:03.342 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:03:03.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:03:03.344 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T16:03:03.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T16:03:03.341+0000 7f4c1b650700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 66469) UID: 1000 2021-02-20T16:03:03.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:03:05.345 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:03:05.345 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:03:05.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:03:05.346 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:03:05.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:03:05.347 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:03:05.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:03:05.350 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T16:03:39.629 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2647882768664125) > weight (0.25) 2021-02-20T16:03:39.630 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4076846972348831) > weight (0.25) 2021-02-20T16:03:39.630 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4182228834565618) > weight (0.25) 2021-02-20T16:03:39.630 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3753577241384366) > weight (0.25) 2021-02-20T16:03:39.631 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T16:05:03.351 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7998461061626563) > weight (0.25) 2021-02-20T16:05:03.352 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4331883284906375) > weight (0.25) 2021-02-20T16:05:03.353 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7638286707264699) > weight (0.25) 2021-02-20T16:05:03.353 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4304058242316722) > weight (0.25) 2021-02-20T16:05:03.353 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T16:05:14.631 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5871307142258851) > weight (0.25) 2021-02-20T16:05:14.632 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7746280112147579) > weight (0.25) 2021-02-20T16:05:14.632 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4057344885411811) > weight (0.25) 2021-02-20T16:05:14.632 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T16:05:14.633 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T16:05:14.634 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T16:05:14.635 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T16:05:14.631+0000 7f6769136700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 65629) UID: 1000 2021-02-20T16:05:15.038 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:05:17.635 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T16:05:17.635 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:05:17.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:05:17.636 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T16:05:17.636 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T16:05:17.637 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T16:05:17.640 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T16:05:17.640 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T16:06:13.354 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4027561113753778) > weight (0.25) 2021-02-20T16:06:13.354 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.740746925492643) > weight (0.25) 2021-02-20T16:06:13.355 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8408049796537936) > weight (0.25) 2021-02-20T16:06:13.355 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6004802729972423) > weight (0.25) 2021-02-20T16:06:13.355 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T16:06:15.899 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T16:06:15.894+0000 7fb533c45700 -1 reset not still connected to 0x558b8d9fd5f0 2021-02-20T16:07:01.641 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5167599292436409) > weight (0.25) 2021-02-20T16:07:01.642 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3327228909977057) > weight (0.25) 2021-02-20T16:07:01.642 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8574334301775902) > weight (0.25) 2021-02-20T16:07:01.643 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8250169478337558) > weight (0.25) 2021-02-20T16:07:01.643 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T16:08:05.356 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5550359270849693) > weight (0.25) 2021-02-20T16:08:05.357 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3027610869882611) > weight (0.25) 2021-02-20T16:08:05.358 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6991444148948007) > weight (0.25) 2021-02-20T16:08:05.358 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5659801178084163) > weight (0.25) 2021-02-20T16:08:05.359 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T16:08:23.644 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5739622299180349) > weight (0.25) 2021-02-20T16:08:23.645 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8852641401275886) > weight (0.25) 2021-02-20T16:08:23.645 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T16:08:23.646 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T16:08:23.646 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T16:08:23.647 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T16:08:23.644+0000 7fbe8e4a7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 67270) UID: 1000 2021-02-20T16:08:23.848 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:08:25.647 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T16:08:25.648 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:08:25.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:08:25.649 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T16:08:25.649 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T16:08:25.650 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T16:08:25.653 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T16:08:25.653 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T16:09:56.654 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.29215964717422394) > weight (0.25) 2021-02-20T16:09:56.655 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9823709436198275) > weight (0.25) 2021-02-20T16:09:56.655 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.26725354732522144) > weight (0.25) 2021-02-20T16:09:56.655 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.25984592018569763) > weight (0.25) 2021-02-20T16:09:56.656 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T16:09:59.359 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.28590331502727784) > weight (0.25) 2021-02-20T16:09:59.360 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8574677696260604) > weight (0.25) 2021-02-20T16:09:59.360 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6520543335794707) > weight (0.25) 2021-02-20T16:09:59.360 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5819107497373748) > weight (0.25) 2021-02-20T16:09:59.361 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T16:11:22.362 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9451264050644477) > weight (0.25) 2021-02-20T16:11:22.363 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9176281915284894) > weight (0.25) 2021-02-20T16:11:22.363 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T16:11:22.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T16:11:22.365 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T16:11:22.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T16:11:22.361+0000 7f523661f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 67361) UID: 1000 2021-02-20T16:11:22.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:11:23.657 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2502687605207524) > weight (0.25) 2021-02-20T16:11:23.658 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.36052696343199253) > weight (0.25) 2021-02-20T16:11:23.658 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2680570634359656) > weight (0.25) 2021-02-20T16:11:23.658 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5002559123958525) > weight (0.25) 2021-02-20T16:11:23.658 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T16:11:29.365 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T16:11:29.366 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:11:29.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:11:29.367 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T16:11:29.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T16:11:29.367 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T16:11:29.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T16:11:29.370 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T16:12:28.660 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7394630349115808) > weight (0.25) 2021-02-20T16:12:28.661 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6513395522629175) > weight (0.25) 2021-02-20T16:12:28.661 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4421347076483615) > weight (0.25) 2021-02-20T16:12:28.662 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T16:12:28.663 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T16:12:28.663 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T16:12:28.665 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T16:12:28.660+0000 7fb6c659e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 68279) UID: 1000 2021-02-20T16:12:28.865 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:12:32.664 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T16:12:32.665 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:12:32.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:12:32.665 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T16:12:32.666 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T16:12:32.666 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T16:12:32.669 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T16:12:32.669 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T16:12:41.371 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6723275406734269) > weight (0.25) 2021-02-20T16:12:41.372 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9588032743553153) > weight (0.25) 2021-02-20T16:12:41.372 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8510992083144422) > weight (0.25) 2021-02-20T16:12:41.372 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.30831706882698084) > weight (0.25) 2021-02-20T16:12:41.372 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T16:13:54.374 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3756265062223775) > weight (0.25) 2021-02-20T16:13:54.375 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.34690919847013724) > weight (0.25) 2021-02-20T16:13:54.375 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5394159585479757) > weight (0.25) 2021-02-20T16:13:54.375 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7693661874957831) > weight (0.25) 2021-02-20T16:13:54.375 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T16:14:19.670 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7093846649678432) > weight (0.25) 2021-02-20T16:14:19.671 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4657057245086681) > weight (0.25) 2021-02-20T16:14:19.671 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.41072391398437946) > weight (0.25) 2021-02-20T16:14:19.671 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8633382801042988) > weight (0.25) 2021-02-20T16:14:19.672 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T16:15:53.377 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:15:53.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:15:53.379 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T16:15:53.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T16:15:53.376+0000 7f6a9ff2c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 68185) UID: 1000 2021-02-20T16:15:53.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:16:00.380 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:16:00.381 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:16:00.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:16:00.382 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:16:00.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:16:00.383 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:16:00.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:16:00.386 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T16:16:00.673 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.987328414881181) > weight (0.25) 2021-02-20T16:16:00.673 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.43656875983663546) > weight (0.25) 2021-02-20T16:16:00.673 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T16:16:00.674 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T16:16:00.674 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T16:16:00.675 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T16:16:00.672+0000 7f0a220c6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 68371) UID: 1000 2021-02-20T16:16:00.876 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:16:06.675 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T16:16:06.676 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:16:06.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:16:06.676 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T16:16:06.677 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T16:16:06.677 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T16:16:06.680 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T16:16:06.681 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T16:16:45.884 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T16:16:45.878+0000 7fb533c45700 -1 reset not still connected to 0x558b8d9fd5f0 2021-02-20T16:17:28.386 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:17:28.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:17:28.388 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T16:17:28.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T16:17:28.386+0000 7f5f4fecb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 68648) UID: 1000 2021-02-20T16:17:28.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:17:31.389 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:17:31.390 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:17:31.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:17:31.390 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:17:31.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:17:31.391 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:17:31.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:17:31.395 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T16:17:37.681 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:17:37.682 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:17:37.682 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T16:17:37.683 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T16:17:37.678+0000 7f5a18db8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 67820) UID: 1000 2021-02-20T16:17:38.084 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:17:45.683 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:17:45.684 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:17:45.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:17:45.684 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:17:45.685 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:17:45.685 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:17:45.688 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:17:45.688 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T16:18:50.690 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5243178457074942) > weight (0.25) 2021-02-20T16:18:50.691 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:18:50.692 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:18:50.692 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T16:18:50.693 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T16:18:50.685+0000 7f119b77b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 68003) UID: 1000 2021-02-20T16:18:51.093 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:18:56.396 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:18:56.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:18:56.397 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T16:18:56.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T16:18:56.389+0000 7ff7edad2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 68833) UID: 1000 2021-02-20T16:18:56.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:18:56.693 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:18:56.694 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:18:56.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:18:56.695 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:18:56.695 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:18:56.696 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:18:56.699 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:18:56.700 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T16:19:03.398 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:19:03.398 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:19:03.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:19:03.399 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:19:03.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:19:03.399 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:19:03.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:19:03.402 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T16:20:32.701 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7659570323138598) > weight (0.25) 2021-02-20T16:20:32.701 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:20:32.702 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:20:32.702 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T16:20:32.703 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T16:20:32.699+0000 7f94eed02700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 69015) UID: 1000 2021-02-20T16:20:32.904 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:20:39.703 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:20:39.703 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:20:39.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:20:39.704 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:20:39.704 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:20:39.705 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:20:39.707 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:20:39.708 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T16:20:54.403 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7465251045198757) > weight (0.25) 2021-02-20T16:20:54.404 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:20:54.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:20:54.405 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T16:20:54.405 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:20:54.405 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:20:54.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T16:20:54.399+0000 7f9dfcb50700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 68094) UID: 1000 2021-02-20T16:20:54.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:21:00.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:21:00.508 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:21:00.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:21:00.509 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:21:00.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:21:00.515 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T16:21:54.708 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.35845147739076655) > weight (0.25) 2021-02-20T16:21:54.709 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.846126837833362) > weight (0.25) 2021-02-20T16:21:54.709 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.955590365661146) > weight (0.25) 2021-02-20T16:21:54.710 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T16:21:54.710 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T16:21:54.711 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:21:54.712 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T16:21:54.706+0000 7fbe703b0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 68554) UID: 1000 2021-02-20T16:21:55.112 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:21:59.712 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T16:21:59.713 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:21:59.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:21:59.713 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T16:21:59.714 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T16:21:59.714 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T16:21:59.717 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T16:21:59.717 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T16:22:16.515 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:22:16.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:22:16.517 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T16:22:16.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T16:22:16.510+0000 7fc7b9cf2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 69106) UID: 1000 2021-02-20T16:22:16.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:22:20.518 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:22:20.518 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:22:20.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:22:20.519 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:22:20.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:22:20.520 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:22:20.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:22:20.523 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T16:23:18.718 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7436340887343004) > weight (0.25) 2021-02-20T16:23:18.719 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.41075494331846074) > weight (0.25) 2021-02-20T16:23:18.719 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T16:23:18.719 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T16:23:18.720 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T16:23:18.721 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T16:23:18.712+0000 7fa20579b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 68739) UID: 1000 2021-02-20T16:23:19.121 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:23:20.720 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T16:23:20.721 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:23:20.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:23:20.722 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T16:23:20.722 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T16:23:20.722 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T16:23:20.725 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T16:23:20.725 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T16:23:42.524 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5836255323973428) > weight (0.25) 2021-02-20T16:23:42.524 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8258176261377589) > weight (0.25) 2021-02-20T16:23:42.524 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5468491381528616) > weight (0.25) 2021-02-20T16:23:42.525 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.2739714474000793) > weight (0.25) 2021-02-20T16:23:42.525 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T16:24:48.727 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:24:48.728 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:24:48.728 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:24:48.729 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T16:24:48.723+0000 7f5e73c47700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 68924) UID: 1000 2021-02-20T16:24:48.929 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:24:53.728 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:24:53.729 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:24:53.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:24:53.730 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:24:53.730 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:24:53.730 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:24:53.733 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:24:53.734 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T16:25:03.525 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:25:03.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:25:03.526 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T16:25:03.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T16:25:03.523+0000 7f1192b14700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 69472) UID: 1000 2021-02-20T16:25:03.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:25:07.527 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:25:07.528 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:25:07.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:25:07.529 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:25:07.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:25:07.529 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:25:07.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:25:07.532 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T16:26:06.734 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.778578418912878) > weight (0.25) 2021-02-20T16:26:06.735 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9173961601822698) > weight (0.25) 2021-02-20T16:26:06.735 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6009810673762396) > weight (0.25) 2021-02-20T16:26:06.735 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T16:26:06.736 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T16:26:06.736 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T16:26:06.737 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T16:26:06.730+0000 7fa08c5ee700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 69381) UID: 1000 2021-02-20T16:26:07.138 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:26:07.737 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T16:26:07.737 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:26:07.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:26:07.738 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T16:26:07.738 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T16:26:07.739 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T16:26:07.741 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T16:26:07.742 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T16:27:01.533 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:27:01.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:27:01.535 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T16:27:01.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T16:27:01.533+0000 7f3bbe7fc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 69748) UID: 1000 2021-02-20T16:27:01.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:27:03.536 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:27:03.536 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:27:03.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:27:03.537 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:27:03.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:27:03.538 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:27:03.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:27:03.540 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T16:27:31.743 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6476051044473337) > weight (0.25) 2021-02-20T16:27:31.744 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5416889035361656) > weight (0.25) 2021-02-20T16:27:31.744 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9705828556904463) > weight (0.25) 2021-02-20T16:27:31.744 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.44302133722128045) > weight (0.25) 2021-02-20T16:27:31.745 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T16:28:40.541 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:28:40.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:28:40.543 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T16:28:40.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T16:28:40.539+0000 7fde90558700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 69930) UID: 1000 2021-02-20T16:28:40.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:28:43.544 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:28:43.545 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:28:43.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:28:43.546 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:28:43.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:28:43.546 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:28:43.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:28:43.550 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T16:29:24.746 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:29:24.747 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:29:24.748 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T16:29:24.749 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T16:29:24.742+0000 7fd64be29700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 69654) UID: 1000 2021-02-20T16:29:24.949 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:29:25.748 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:29:25.749 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:29:25.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:29:25.750 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:29:25.750 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:29:25.750 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:29:25.753 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:29:25.753 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T16:30:01.550 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:30:01.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:30:01.552 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T16:30:01.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T16:30:01.550+0000 7f5c2d259700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 70022) UID: 1000 2021-02-20T16:30:01.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:30:08.553 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:30:08.553 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:30:08.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:30:08.554 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:30:08.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:30:08.555 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:30:08.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:30:08.559 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T16:31:13.559 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:31:13.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:31:13.568 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T16:31:13.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T16:31:13.564+0000 7fd0ca1af700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 70206) UID: 1000 2021-02-20T16:31:13.754 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5331838348717233) > weight (0.25) 2021-02-20T16:31:13.755 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5879425276688495) > weight (0.25) 2021-02-20T16:31:13.755 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4523856120968708) > weight (0.25) 2021-02-20T16:31:13.755 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.875597747090769) > weight (0.25) 2021-02-20T16:31:13.756 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T16:31:13.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:31:16.569 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:31:16.569 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:31:16.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:31:16.570 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:31:16.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:31:16.571 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:31:16.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:31:16.573 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T16:33:00.575 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:33:00.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:33:00.585 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T16:33:00.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T16:33:00.579+0000 7f5b90561700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 70297) UID: 1000 2021-02-20T16:33:00.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:33:04.586 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:33:04.586 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:33:04.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:33:04.587 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:33:04.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:33:04.588 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:33:04.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:33:04.591 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T16:33:05.756 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2980290136305528) > weight (0.25) 2021-02-20T16:33:05.757 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7665725430658674) > weight (0.25) 2021-02-20T16:33:05.757 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T16:33:05.757 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T16:33:05.758 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:33:05.758 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T16:33:05.754+0000 7ff7dbf3f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 69563) UID: 1000 2021-02-20T16:33:05.959 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:33:10.759 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T16:33:10.759 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:33:10.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:33:10.760 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T16:33:10.760 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T16:33:10.760 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T16:33:10.763 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T16:33:10.763 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T16:34:35.592 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.39755757644808176) > weight (0.25) 2021-02-20T16:34:35.603 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5214363805984074) > weight (0.25) 2021-02-20T16:34:35.603 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T16:34:35.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T16:34:35.604 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T16:34:35.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T16:34:35.597+0000 7f0771840700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 68463) UID: 1000 2021-02-20T16:34:35.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:34:39.605 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T16:34:39.606 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:34:39.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:34:39.606 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T16:34:39.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T16:34:39.607 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T16:34:39.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T16:34:39.610 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T16:35:09.765 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.34968424237854634) > weight (0.25) 2021-02-20T16:35:09.776 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:35:09.776 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:35:09.778 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T16:35:09.778 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T16:35:09.768+0000 7f128b8c9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 69199) UID: 1000 2021-02-20T16:35:10.178 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:35:17.779 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:35:17.779 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:35:17.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:35:17.781 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:35:17.781 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:35:17.781 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:35:17.784 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:35:17.785 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T16:36:30.612 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.33030276607956943) > weight (0.25) 2021-02-20T16:36:30.615 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:36:30.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:36:30.617 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T16:36:30.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T16:36:30.611+0000 7fe425837700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 69290) UID: 1000 2021-02-20T16:36:30.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:36:36.618 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:36:36.618 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:36:36.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:36:36.619 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:36:36.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:36:36.619 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:36:36.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:36:36.622 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T16:37:13.786 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7057075932046611) > weight (0.25) 2021-02-20T16:37:13.794 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7767681895100992) > weight (0.25) 2021-02-20T16:37:13.795 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5637383080600746) > weight (0.25) 2021-02-20T16:37:13.795 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9244968785347575) > weight (0.25) 2021-02-20T16:37:13.795 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T16:37:56.623 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7088535124063393) > weight (0.25) 2021-02-20T16:37:56.630 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.38378940266163486) > weight (0.25) 2021-02-20T16:37:56.630 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9577886728421489) > weight (0.25) 2021-02-20T16:37:56.631 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8716443399861329) > weight (0.25) 2021-02-20T16:37:56.631 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T16:38:30.796 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5198336346681969) > weight (0.25) 2021-02-20T16:38:30.797 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5087798646888211) > weight (0.25) 2021-02-20T16:38:30.797 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T16:38:30.798 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T16:38:30.799 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T16:38:30.800 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T16:38:30.797+0000 7f9c03ae0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 70479) UID: 1000 2021-02-20T16:38:31.000 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:38:38.800 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T16:38:38.807 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:38:38.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:38:38.808 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T16:38:38.808 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T16:38:38.809 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T16:38:38.813 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T16:38:38.813 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T16:39:24.632 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9589749209169053) > weight (0.25) 2021-02-20T16:39:24.640 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:39:24.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:39:24.642 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T16:39:24.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T16:39:24.636+0000 7f3934e01700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 70755) UID: 1000 2021-02-20T16:39:24.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:39:30.643 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:39:30.644 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:39:30.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:39:30.645 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:39:30.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:39:30.646 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:39:30.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:39:30.649 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T16:39:44.814 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7685064105182785) > weight (0.25) 2021-02-20T16:39:44.815 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:39:44.815 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:39:44.816 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T16:39:44.816 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:39:44.816 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:39:44.817 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T16:39:44.812+0000 7f4ee78f2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 70664) UID: 1000 2021-02-20T16:39:45.017 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:39:50.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:39:50.919 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:39:50.919 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:39:50.919 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:39:50.922 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:39:50.922 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T16:40:50.650 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:40:50.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:40:50.652 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:40:50.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T16:40:50.647+0000 7f775b462700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 70388) UID: 1000 2021-02-20T16:40:50.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:40:55.653 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:40:55.654 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:40:55.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:40:55.655 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:40:55.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:40:55.656 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:40:55.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:40:55.660 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T16:41:15.924 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9533383817240972) > weight (0.25) 2021-02-20T16:41:15.925 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:41:15.926 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:41:15.926 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T16:41:15.927 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T16:41:15.922+0000 7fbe1cce1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 71029) UID: 1000 2021-02-20T16:41:16.127 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:41:22.927 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:41:22.927 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:41:22.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:41:22.928 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:41:22.928 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:41:22.929 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:41:22.932 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:41:22.932 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T16:42:10.662 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:42:10.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:42:10.664 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T16:42:10.664 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:42:10.664 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:42:10.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T16:42:10.657+0000 7f5c0138e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 71121) UID: 1000 2021-02-20T16:42:10.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:42:16.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:42:16.766 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:42:16.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:42:16.767 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:42:16.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:42:16.770 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T16:43:18.933 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9372465374167075) > weight (0.25) 2021-02-20T16:43:18.934 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:43:18.935 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:43:18.935 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T16:43:18.936 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T16:43:18.932+0000 7ff13a37b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 71212) UID: 1000 2021-02-20T16:43:19.136 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:43:26.936 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:43:26.937 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:43:26.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:43:26.937 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:43:26.938 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:43:26.938 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:43:26.941 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:43:26.941 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T16:44:02.771 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6746025323632017) > weight (0.25) 2021-02-20T16:44:02.772 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:44:02.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:44:02.773 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T16:44:02.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T16:44:02.771+0000 7fe8f187a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 70938) UID: 1000 2021-02-20T16:44:02.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:44:06.774 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:44:06.774 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:44:06.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:44:06.775 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:44:06.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:44:06.776 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:44:06.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:44:06.779 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T16:44:41.942 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:44:41.943 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:44:41.944 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T16:44:41.945 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T16:44:41.939+0000 7fb9a101c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 70114) UID: 1000 2021-02-20T16:44:42.145 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:44:49.945 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:44:49.946 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:44:49.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:44:49.946 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:44:49.947 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:44:49.947 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:44:49.950 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:44:49.950 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T16:45:31.781 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6520992978427304) > weight (0.25) 2021-02-20T16:45:31.781 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.509626319484399) > weight (0.25) 2021-02-20T16:45:31.781 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5956370116813775) > weight (0.25) 2021-02-20T16:45:31.782 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5764417078338046) > weight (0.25) 2021-02-20T16:45:31.782 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T16:46:30.952 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6847880236422076) > weight (0.25) 2021-02-20T16:46:30.952 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.890336754956118) > weight (0.25) 2021-02-20T16:46:30.953 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.43360563057271806) > weight (0.25) 2021-02-20T16:46:30.953 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T16:46:30.954 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T16:46:30.954 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:46:30.955 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T16:46:30.949+0000 7f34870fe700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 69839) UID: 1000 2021-02-20T16:46:31.356 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:46:35.955 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T16:46:35.956 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:46:35.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:46:35.957 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T16:46:35.958 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T16:46:35.958 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T16:46:35.962 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T16:46:35.962 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T16:46:58.783 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9814069524617026) > weight (0.25) 2021-02-20T16:46:58.784 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6284564927027368) > weight (0.25) 2021-02-20T16:46:58.784 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.781827060450323) > weight (0.25) 2021-02-20T16:46:58.784 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T16:46:58.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T16:46:58.785 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T16:46:58.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T16:46:58.784+0000 7f97859b0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 66088) UID: 1000 2021-02-20T16:46:59.042 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T16:46:59.038+0000 7fb533c45700 -1 reset not still connected to 0x558b8917c680 2021-02-20T16:46:59.044 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T16:46:59.038+0000 7fb533c45700 -1 reset not still connected to 0x558b94652700 2021-02-20T16:46:59.045 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T16:46:59.038+0000 7fb533c45700 -1 reset not still connected to 0x558b948cfcf0 2021-02-20T16:46:59.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:47:01.786 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T16:47:01.786 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:47:01.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:47:01.787 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T16:47:01.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T16:47:01.788 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T16:47:01.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T16:47:01.791 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T16:47:39.964 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9672035483990052) > weight (0.25) 2021-02-20T16:47:39.964 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:47:39.965 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:47:39.966 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T16:47:39.966 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T16:47:39.964+0000 7fe0af526700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 71394) UID: 1000 2021-02-20T16:47:40.167 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:47:47.967 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:47:47.967 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:47:47.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:47:47.968 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:47:47.968 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:47:47.969 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:47:47.972 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:47:47.972 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T16:48:48.792 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.42298739357095494) > weight (0.25) 2021-02-20T16:48:48.792 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7457141275426935) > weight (0.25) 2021-02-20T16:48:48.793 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.390568625733268) > weight (0.25) 2021-02-20T16:48:48.793 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4357083560785021) > weight (0.25) 2021-02-20T16:48:48.793 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T16:49:22.974 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.43369369244774825) > weight (0.25) 2021-02-20T16:49:22.974 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.40287079422957695) > weight (0.25) 2021-02-20T16:49:22.974 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.26152333592533095) > weight (0.25) 2021-02-20T16:49:22.975 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T16:49:22.976 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T16:49:22.976 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T16:49:22.977 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T16:49:22.970+0000 7f3b3e12e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 71671) UID: 1000 2021-02-20T16:49:23.177 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:49:28.977 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T16:49:28.978 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:49:28.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:49:28.979 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T16:49:28.979 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T16:49:28.980 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T16:49:28.983 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T16:49:28.983 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T16:49:54.794 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9018382592950562) > weight (0.25) 2021-02-20T16:49:54.795 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8668867582232352) > weight (0.25) 2021-02-20T16:49:54.795 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T16:49:54.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T16:49:54.796 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T16:49:54.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T16:49:54.789+0000 7f0c510b5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 70570) UID: 1000 2021-02-20T16:49:55.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:49:56.797 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T16:49:56.797 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:49:56.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:49:56.798 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T16:49:56.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T16:49:56.799 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T16:49:56.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T16:49:56.802 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T16:51:19.984 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:51:19.986 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:51:19.986 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T16:51:19.987 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T16:51:19.984+0000 7f038fe7e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 71577) UID: 1000 2021-02-20T16:51:20.388 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:51:23.988 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:51:23.988 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:51:23.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:51:23.989 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:51:23.989 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:51:23.990 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:51:23.992 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:51:23.993 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T16:51:50.803 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.47208082499839066) > weight (0.25) 2021-02-20T16:51:50.803 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:51:50.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:51:50.805 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T16:51:50.805 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:51:50.805 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:51:50.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T16:51:50.803+0000 7f3da1e9f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 71485) UID: 1000 2021-02-20T16:51:51.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:51:56.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:51:56.907 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:51:56.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:51:56.907 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:51:56.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:51:56.911 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T16:52:42.993 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6897103879550674) > weight (0.25) 2021-02-20T16:52:42.994 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:52:42.995 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:52:42.995 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:52:42.996 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T16:52:42.990+0000 7f8ed954f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 71861) UID: 1000 2021-02-20T16:52:43.196 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:52:47.996 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:52:47.997 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:52:47.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:52:47.997 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:52:47.998 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:52:47.998 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:52:48.001 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:52:48.001 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T16:53:50.912 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8121266669381526) > weight (0.25) 2021-02-20T16:53:50.913 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5375842444241685) > weight (0.25) 2021-02-20T16:53:50.913 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7519645777726333) > weight (0.25) 2021-02-20T16:53:50.913 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T16:53:50.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T16:53:50.914 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T16:53:50.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T16:53:50.913+0000 7f0919f67700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 71762) UID: 1000 2021-02-20T16:53:51.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:53:52.916 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T16:53:52.916 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:53:52.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:53:52.917 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T16:53:52.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T16:53:52.919 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T16:53:52.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T16:53:52.923 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T16:53:53.002 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:53:53.003 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:53:53.003 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T16:53:53.003 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:53:53.004 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:53:53.004 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T16:53:52.997+0000 7f5b91a2b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 72146) UID: 1000 2021-02-20T16:53:53.204 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:53:59.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:53:59.107 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:53:59.107 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:53:59.107 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:53:59.110 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:53:59.111 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T16:54:59.924 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:54:59.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:54:59.926 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:54:59.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T16:54:59.924+0000 7f77afbad700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 71303) UID: 1000 2021-02-20T16:55:00.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:55:04.927 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:55:04.928 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:55:04.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:55:04.929 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:55:04.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:55:04.929 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:55:04.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:55:04.933 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T16:55:06.112 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:55:06.112 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:55:06.113 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T16:55:06.113 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T16:55:06.112+0000 7f4c7948d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 72518) UID: 1000 2021-02-20T16:55:06.314 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:55:13.113 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:55:13.114 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:55:13.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:55:13.115 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:55:13.115 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:55:13.116 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:55:13.118 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:55:13.118 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T16:56:22.934 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:56:22.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:56:22.936 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T16:56:22.936 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:56:22.936 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:56:22.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T16:56:22.931+0000 7fe2cea82700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 72612) UID: 1000 2021-02-20T16:56:23.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:56:29.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:56:29.038 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:56:29.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:56:29.039 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:56:29.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:56:29.042 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T16:56:58.119 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4136541000029894) > weight (0.25) 2021-02-20T16:56:58.120 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:56:58.121 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:56:58.121 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T16:56:58.122 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T16:56:58.118+0000 7f65c7e28700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 72328) UID: 1000 2021-02-20T16:56:58.322 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:57:04.122 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:57:04.123 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:57:04.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:57:04.124 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:57:04.124 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:57:04.124 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:57:04.127 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:57:04.127 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T16:58:02.042 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5447554794147706) > weight (0.25) 2021-02-20T16:58:02.043 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8317437047953861) > weight (0.25) 2021-02-20T16:58:02.043 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.27279812677757165) > weight (0.25) 2021-02-20T16:58:02.044 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T16:58:02.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T16:58:02.045 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T16:58:02.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T16:58:02.041+0000 7f5a2bfa5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 72419) UID: 1000 2021-02-20T16:58:02.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:58:08.045 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T16:58:08.046 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:58:08.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:58:08.047 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T16:58:08.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T16:58:08.048 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T16:58:08.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T16:58:08.051 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T16:58:40.128 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:58:40.130 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:58:40.130 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T16:58:40.131 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T16:58:40.128+0000 7faf73569700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 72703) UID: 1000 2021-02-20T16:58:40.332 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:58:43.130 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:58:43.131 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:58:43.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:58:43.132 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:58:43.132 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:58:43.132 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:58:43.135 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:58:43.135 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T16:59:54.052 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7788200491857054) > weight (0.25) 2021-02-20T16:59:54.053 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:59:54.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:59:54.054 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T16:59:54.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T16:59:54.051+0000 7ff77d445700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 72237) UID: 1000 2021-02-20T16:59:54.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:59:57.055 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:59:57.056 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:59:57.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:59:57.057 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:59:57.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:59:57.057 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:59:57.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:59:57.061 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T17:00:07.136 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.29884080448007944) > weight (0.25) 2021-02-20T17:00:07.137 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.554691849953147) > weight (0.25) 2021-02-20T17:00:07.137 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T17:00:07.138 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T17:00:07.138 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T17:00:07.139 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T17:00:07.135+0000 7f9be6802700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 70847) UID: 1000 2021-02-20T17:00:07.540 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:00:16.140 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T17:00:16.140 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:00:16.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:00:16.141 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T17:00:16.142 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T17:00:16.142 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T17:00:16.145 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T17:00:16.145 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T17:00:58.062 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.25524533023389284) > weight (0.25) 2021-02-20T17:00:58.062 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8770261907153258) > weight (0.25) 2021-02-20T17:00:58.063 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T17:00:58.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T17:00:58.064 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T17:00:58.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T17:00:58.062+0000 7f87a1a48700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 72043) UID: 1000 2021-02-20T17:00:58.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:01:07.065 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T17:01:07.066 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:01:07.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:01:07.066 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T17:01:07.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T17:01:07.067 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T17:01:07.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T17:01:07.070 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T17:01:25.147 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2520320801692757) > weight (0.25) 2021-02-20T17:01:25.147 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9867186744901073) > weight (0.25) 2021-02-20T17:01:25.147 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T17:01:25.148 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T17:01:25.148 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T17:01:25.149 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T17:01:25.146+0000 7fcdff4ff700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 73259) UID: 1000 2021-02-20T17:01:25.350 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:01:28.149 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T17:01:28.149 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:01:28.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:01:28.150 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T17:01:28.150 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T17:01:28.151 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T17:01:28.153 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T17:01:28.154 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T17:02:34.420 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T17:02:34.394+0000 7fb533c45700 -1 reset not still connected to 0x558b948cfe90 2021-02-20T17:02:49.071 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5623488370843546) > weight (0.25) 2021-02-20T17:02:49.072 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T17:02:49.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T17:02:49.073 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T17:02:49.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T17:02:49.072+0000 7fe361906700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 73159) UID: 1000 2021-02-20T17:02:49.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:02:52.155 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6772273085420365) > weight (0.25) 2021-02-20T17:02:52.156 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5083883750311973) > weight (0.25) 2021-02-20T17:02:52.156 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5119019507230229) > weight (0.25) 2021-02-20T17:02:52.156 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T17:02:52.157 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T17:02:52.157 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T17:02:52.158 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T17:02:52.156+0000 7f6981242700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 71952) UID: 1000 2021-02-20T17:02:52.358 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:02:54.073 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T17:02:54.074 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:02:54.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:02:54.075 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T17:02:54.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T17:02:54.075 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T17:02:54.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T17:02:54.078 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T17:02:59.158 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T17:02:59.158 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:02:59.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:02:59.159 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T17:02:59.159 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T17:02:59.159 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T17:02:59.162 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T17:02:59.163 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T17:04:04.079 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4143826291486029) > weight (0.25) 2021-02-20T17:04:04.080 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T17:04:04.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T17:04:04.081 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T17:04:04.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T17:04:04.079+0000 7f3e6cec1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 73532) UID: 1000 2021-02-20T17:04:04.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:04:08.082 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T17:04:08.083 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:04:08.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:04:08.084 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T17:04:08.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T17:04:08.085 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T17:04:08.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T17:04:08.089 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T17:04:17.163 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6254316769403502) > weight (0.25) 2021-02-20T17:04:17.164 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3047606116780721) > weight (0.25) 2021-02-20T17:04:17.164 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T17:04:17.165 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T17:04:17.165 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T17:04:17.166 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T17:04:17.163+0000 7f270b224700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 73441) UID: 1000 2021-02-20T17:04:17.366 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:04:22.167 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T17:04:22.167 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:04:22.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:04:22.168 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T17:04:22.168 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T17:04:22.168 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T17:04:22.171 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T17:04:22.171 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T17:05:29.090 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:05:29.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:05:29.092 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T17:05:29.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T17:05:29.085+0000 7fd98c00b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 72794) UID: 1000 2021-02-20T17:05:29.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:05:31.093 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:05:31.093 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:05:31.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:05:31.094 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:05:31.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:05:31.096 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:05:31.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:05:31.100 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T17:05:40.172 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9438105834492664) > weight (0.25) 2021-02-20T17:05:40.173 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4208967204458982) > weight (0.25) 2021-02-20T17:05:40.173 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T17:05:40.173 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T17:05:40.174 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T17:05:40.174 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T17:05:40.174 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:05:40.175 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T17:05:40.169+0000 7fed63176700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 73805) UID: 1000 2021-02-20T17:05:40.375 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:05:46.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:05:46.276 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T17:05:46.276 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T17:05:46.276 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T17:05:46.279 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T17:05:46.279 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T17:06:38.601 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T17:06:38.595+0000 7fb533c45700 -1 reset not still connected to 0x558b948cd450 2021-02-20T17:06:54.101 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.35229381041395236) > weight (0.25) 2021-02-20T17:06:54.102 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.627432592967288) > weight (0.25) 2021-02-20T17:06:54.102 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.766867136000985) > weight (0.25) 2021-02-20T17:06:54.102 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T17:06:54.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T17:06:54.103 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T17:06:54.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T17:06:54.100+0000 7fb352b20700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 72977) UID: 1000 2021-02-20T17:06:54.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:07:00.105 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T17:07:00.105 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:07:00.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:07:00.106 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T17:07:00.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T17:07:00.107 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T17:07:00.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T17:07:00.110 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T17:07:19.280 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6871264569783845) > weight (0.25) 2021-02-20T17:07:19.281 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5122479296879753) > weight (0.25) 2021-02-20T17:07:19.281 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T17:07:19.281 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T17:07:19.282 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T17:07:19.282 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T17:07:19.280+0000 7f7394fd9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 73991) UID: 1000 2021-02-20T17:07:19.483 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:07:23.283 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T17:07:23.283 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:07:23.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:07:23.284 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T17:07:23.284 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T17:07:23.285 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T17:07:23.287 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T17:07:23.288 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T17:08:05.111 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:08:05.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:08:05.113 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T17:08:05.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T17:08:05.111+0000 7f1d22d93700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 73900) UID: 1000 2021-02-20T17:08:05.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:08:07.114 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:08:07.114 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:08:07.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:08:07.115 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:08:07.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:08:07.116 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:08:07.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:08:07.119 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T17:09:11.289 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.717474065307615) > weight (0.25) 2021-02-20T17:09:11.289 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7344431920394152) > weight (0.25) 2021-02-20T17:09:11.290 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5391182486124967) > weight (0.25) 2021-02-20T17:09:11.290 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8840372212884036) > weight (0.25) 2021-02-20T17:09:11.290 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T17:09:20.120 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:09:20.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:09:20.122 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T17:09:20.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T17:09:20.117+0000 7f4cc5e9c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 74272) UID: 1000 2021-02-20T17:09:20.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:09:25.123 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:09:25.123 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:09:25.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:09:25.124 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:09:25.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:09:25.125 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:09:25.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:09:25.128 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T17:10:54.291 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8292020176411344) > weight (0.25) 2021-02-20T17:10:54.292 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9452331869947304) > weight (0.25) 2021-02-20T17:10:54.293 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9504923995653182) > weight (0.25) 2021-02-20T17:10:54.293 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T17:10:54.294 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T17:10:54.294 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T17:10:54.295 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T17:10:54.292+0000 7fddd7d4f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 73623) UID: 1000 2021-02-20T17:10:54.495 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:11:01.295 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T17:11:01.295 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:11:01.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:11:01.296 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T17:11:01.297 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T17:11:01.297 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T17:11:01.300 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T17:11:01.300 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T17:11:18.129 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.25819573594394585) > weight (0.25) 2021-02-20T17:11:18.130 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3066991396914984) > weight (0.25) 2021-02-20T17:11:18.130 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T17:11:18.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T17:11:18.131 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T17:11:18.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T17:11:18.127+0000 7fdb38263700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 73350) UID: 1000 2021-02-20T17:11:18.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:11:22.688 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T17:11:22.684+0000 7fb533c45700 -1 reset not still connected to 0x558b95d3d1e0 2021-02-20T17:11:22.689 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T17:11:22.684+0000 7fb533c45700 -1 reset not still connected to 0x558b95d3e8a0 2021-02-20T17:11:22.689 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T17:11:22.684+0000 7fb533c45700 -1 reset not still connected to 0x558b9a127790 2021-02-20T17:11:27.132 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T17:11:27.132 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:11:27.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:11:27.133 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T17:11:27.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T17:11:27.133 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T17:11:27.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T17:11:27.137 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T17:12:15.301 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7822644920030349) > weight (0.25) 2021-02-20T17:12:15.302 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7400351937080439) > weight (0.25) 2021-02-20T17:12:15.302 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7903573220504171) > weight (0.25) 2021-02-20T17:12:15.303 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.972464218040724) > weight (0.25) 2021-02-20T17:12:15.303 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T17:12:45.137 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:12:45.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:12:45.139 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T17:12:45.139 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:12:45.139 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:12:45.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T17:12:45.138+0000 7f5dfd192700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 74371) UID: 1000 2021-02-20T17:12:45.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:12:51.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:12:51.242 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:12:51.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:12:51.243 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:12:51.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:12:51.246 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T17:13:37.304 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2653699051912731) > weight (0.25) 2021-02-20T17:13:37.305 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6380673740883306) > weight (0.25) 2021-02-20T17:13:37.305 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8506302083941508) > weight (0.25) 2021-02-20T17:13:37.306 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8839513638898239) > weight (0.25) 2021-02-20T17:13:37.306 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T17:14:24.247 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9942793862083875) > weight (0.25) 2021-02-20T17:14:24.248 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.28916088658765204) > weight (0.25) 2021-02-20T17:14:24.248 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.48206367250412663) > weight (0.25) 2021-02-20T17:14:24.248 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.954017401316878) > weight (0.25) 2021-02-20T17:14:24.248 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T17:15:01.307 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8212099717129236) > weight (0.25) 2021-02-20T17:15:01.308 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9660282374799154) > weight (0.25) 2021-02-20T17:15:01.308 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7262149786077725) > weight (0.25) 2021-02-20T17:15:01.308 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T17:15:01.309 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T17:15:01.310 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T17:15:01.311 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T17:15:01.308+0000 7f57662dc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 74463) UID: 1000 2021-02-20T17:15:01.511 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:15:09.311 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T17:15:09.311 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:15:09.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:15:09.312 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T17:15:09.312 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T17:15:09.313 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T17:15:09.316 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T17:15:09.316 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T17:15:33.249 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2546068457425854) > weight (0.25) 2021-02-20T17:15:33.249 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T17:15:33.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T17:15:33.251 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T17:15:33.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T17:15:33.247+0000 7fe7731cb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 73714) UID: 1000 2021-02-20T17:15:33.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:15:37.252 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T17:15:37.252 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:15:37.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:15:37.253 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T17:15:37.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T17:15:37.253 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T17:15:37.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T17:15:37.256 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T17:16:44.258 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4103509477832308) > weight (0.25) 2021-02-20T17:16:44.259 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.37553461974939173) > weight (0.25) 2021-02-20T17:16:44.259 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T17:16:44.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T17:16:44.260 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T17:16:44.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T17:16:44.258+0000 7fee000ec700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 74562) UID: 1000 2021-02-20T17:16:44.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:16:53.261 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T17:16:53.261 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:16:53.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:16:53.262 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T17:16:53.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T17:16:53.263 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T17:16:53.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T17:16:53.266 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T17:17:07.317 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6616880655982221) > weight (0.25) 2021-02-20T17:17:07.318 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6021754408362273) > weight (0.25) 2021-02-20T17:17:07.318 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9387553221395565) > weight (0.25) 2021-02-20T17:17:07.318 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9794342346983155) > weight (0.25) 2021-02-20T17:17:07.318 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T17:18:37.267 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:18:37.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:18:37.269 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T17:18:37.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T17:18:37.268+0000 7fad44abd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 74654) UID: 1000 2021-02-20T17:18:37.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:18:39.319 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.577259282331178) > weight (0.25) 2021-02-20T17:18:39.320 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2673560444807961) > weight (0.25) 2021-02-20T17:18:39.321 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3671617135206905) > weight (0.25) 2021-02-20T17:18:39.321 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3468611406926585) > weight (0.25) 2021-02-20T17:18:39.321 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T17:18:46.270 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:18:46.270 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:18:46.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:18:46.271 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:18:46.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:18:46.272 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:18:46.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:18:46.275 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T17:20:14.277 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5534987686513315) > weight (0.25) 2021-02-20T17:20:14.277 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.508959169131613) > weight (0.25) 2021-02-20T17:20:14.278 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T17:20:14.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T17:20:14.279 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T17:20:14.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T17:20:14.274+0000 7fb4793dd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 74938) UID: 1000 2021-02-20T17:20:14.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:20:21.279 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T17:20:21.280 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:20:21.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:20:21.281 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T17:20:21.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T17:20:21.281 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T17:20:21.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T17:20:21.285 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T17:20:30.322 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.47727695493338873) > weight (0.25) 2021-02-20T17:20:30.323 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T17:20:30.323 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T17:20:30.323 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T17:20:30.324 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T17:20:30.318+0000 7f75af7f0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 72885) UID: 1000 2021-02-20T17:20:30.725 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:20:36.324 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T17:20:36.325 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:20:36.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:20:36.326 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T17:20:36.326 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T17:20:36.326 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T17:20:36.329 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T17:20:36.329 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T17:21:50.331 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8665963821972508) > weight (0.25) 2021-02-20T17:21:50.331 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9246589131465421) > weight (0.25) 2021-02-20T17:21:50.332 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9639551649814797) > weight (0.25) 2021-02-20T17:21:50.332 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T17:21:50.333 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T17:21:50.333 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T17:21:50.334 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T17:21:50.333+0000 7fdb512bc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 74748) UID: 1000 2021-02-20T17:21:50.534 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:21:53.334 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T17:21:53.335 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:21:53.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:21:53.336 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T17:21:53.336 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T17:21:53.336 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T17:21:53.339 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T17:21:53.339 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T17:22:18.286 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8039754010384692) > weight (0.25) 2021-02-20T17:22:18.287 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3923477805966129) > weight (0.25) 2021-02-20T17:22:18.287 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2848558468017789) > weight (0.25) 2021-02-20T17:22:18.287 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T17:22:18.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T17:22:18.288 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T17:22:18.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T17:22:18.288+0000 7f612efb7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 74090) UID: 1000 2021-02-20T17:22:18.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:22:21.289 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T17:22:21.290 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:22:21.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:22:21.290 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T17:22:21.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T17:22:21.291 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T17:22:21.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T17:22:21.294 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T17:22:55.340 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.48331987152188327) > weight (0.25) 2021-02-20T17:22:55.341 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8872402785238347) > weight (0.25) 2021-02-20T17:22:55.341 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8287272323641361) > weight (0.25) 2021-02-20T17:22:55.341 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4756757519810265) > weight (0.25) 2021-02-20T17:22:55.341 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T17:23:28.295 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6047517819159682) > weight (0.25) 2021-02-20T17:23:28.296 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.48505473827750734) > weight (0.25) 2021-02-20T17:23:28.296 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2500134402236134) > weight (0.25) 2021-02-20T17:23:28.297 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6914430850083865) > weight (0.25) 2021-02-20T17:23:28.297 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T17:24:40.298 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6570002999232712) > weight (0.25) 2021-02-20T17:24:40.299 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8961035767300787) > weight (0.25) 2021-02-20T17:24:40.299 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4490766499889618) > weight (0.25) 2021-02-20T17:24:40.299 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.30709504159215184) > weight (0.25) 2021-02-20T17:24:40.300 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T17:24:49.342 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:24:49.343 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:24:49.344 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T17:24:49.345 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T17:24:49.342+0000 7ff849781700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 73068) UID: 1000 2021-02-20T17:24:49.545 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:24:58.344 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:24:58.345 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:24:58.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:24:58.346 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:24:58.346 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:24:58.346 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:24:58.349 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:24:58.349 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T17:26:26.301 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9678538804609111) > weight (0.25) 2021-02-20T17:26:26.301 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.26772373441204245) > weight (0.25) 2021-02-20T17:26:26.302 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8750920283908351) > weight (0.25) 2021-02-20T17:26:26.302 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6723420695311955) > weight (0.25) 2021-02-20T17:26:26.302 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T17:26:42.351 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9383485102929404) > weight (0.25) 2021-02-20T17:26:42.351 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7863269814399275) > weight (0.25) 2021-02-20T17:26:42.352 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6835505280874095) > weight (0.25) 2021-02-20T17:26:42.352 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T17:26:42.353 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T17:26:42.353 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T17:26:42.354 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T17:26:42.352+0000 7f6f49470700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 75317) UID: 1000 2021-02-20T17:26:42.554 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:26:51.353 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T17:26:52.329 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:26:52.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:26:52.331 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T17:26:52.331 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T17:26:52.331 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T17:26:52.335 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T17:26:52.335 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T17:27:36.303 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7383482385973968) > weight (0.25) 2021-02-20T17:27:36.304 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4170363580546713) > weight (0.25) 2021-02-20T17:27:36.304 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T17:27:36.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T17:27:36.305 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T17:27:36.306 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T17:27:36.306 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:27:36.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T17:27:36.303+0000 7f51c33bd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 75134) UID: 1000 2021-02-20T17:27:36.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:27:42.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:27:42.408 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T17:27:42.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T17:27:42.409 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T17:27:42.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T17:27:42.414 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T17:28:25.336 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8991165424098938) > weight (0.25) 2021-02-20T17:28:25.337 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T17:28:25.338 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T17:28:25.338 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T17:28:25.339 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T17:28:25.338+0000 7fd45c586700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 75225) UID: 1000 2021-02-20T17:28:25.539 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:28:29.339 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T17:28:29.340 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:28:29.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:28:29.340 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T17:28:29.341 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T17:28:29.341 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T17:28:29.344 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T17:28:29.344 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T17:28:51.415 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.691564746881575) > weight (0.25) 2021-02-20T17:28:51.416 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T17:28:51.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T17:28:51.417 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T17:28:51.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T17:28:51.413+0000 7f83886b7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 74839) UID: 1000 2021-02-20T17:28:51.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:28:52.417 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T17:28:52.418 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:28:52.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:28:52.419 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T17:28:52.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T17:28:52.420 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T17:28:52.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T17:28:52.423 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T17:30:01.345 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.712044560437534) > weight (0.25) 2021-02-20T17:30:01.353 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5496004652950216) > weight (0.25) 2021-02-20T17:30:01.354 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6112872450375086) > weight (0.25) 2021-02-20T17:30:01.354 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3696452390877304) > weight (0.25) 2021-02-20T17:30:01.354 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T17:30:25.424 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.32275231167774576) > weight (0.25) 2021-02-20T17:30:25.459 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7282595185050637) > weight (0.25) 2021-02-20T17:30:25.460 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6184950781345053) > weight (0.25) 2021-02-20T17:30:25.460 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T17:30:25.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T17:30:25.461 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T17:30:25.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T17:30:25.460+0000 7f7acd17d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 75408) UID: 1000 2021-02-20T17:30:25.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:30:27.712 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T17:30:27.704+0000 7f7308f93700 -1 reset not still connected to 0x55aabd66b110 2021-02-20T17:30:27.712 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T17:30:27.704+0000 7f7308f93700 -1 reset not still connected to 0x55aac2357a00 2021-02-20T17:30:27.713 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T17:30:27.708+0000 7f7308f93700 -1 reset not still connected to 0x55aac7f898e0 2021-02-20T17:30:27.713 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T17:30:27.708+0000 7f7308f93700 -1 reset not still connected to 0x55aac8dd5790 2021-02-20T17:30:27.714 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T17:30:27.708+0000 7f7308f93700 -1 reset not still connected to 0x55aac8dd6970 2021-02-20T17:30:27.714 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T17:30:27.708+0000 7f7308f93700 -1 reset not still connected to 0x55aacd8bf2b0 2021-02-20T17:30:27.714 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T17:30:27.708+0000 7f7308f93700 -1 reset not still connected to 0x55aae72cc680 2021-02-20T17:30:27.715 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T17:30:27.708+0000 7f7308f93700 -1 reset not still connected to 0x55aae72cf330 2021-02-20T17:30:34.462 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T17:30:34.470 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:30:34.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:30:34.471 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T17:30:34.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T17:30:34.472 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T17:30:34.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T17:30:34.475 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T17:31:30.356 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7726949096864623) > weight (0.25) 2021-02-20T17:31:30.363 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9154353116327802) > weight (0.25) 2021-02-20T17:31:30.363 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.280268419581279) > weight (0.25) 2021-02-20T17:31:30.364 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8838111575256719) > weight (0.25) 2021-02-20T17:31:30.364 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T17:32:23.476 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:32:23.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:32:23.485 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T17:32:23.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T17:32:23.482+0000 7f691bdb1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 75041) UID: 1000 2021-02-20T17:32:23.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:32:28.486 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:32:28.486 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:32:28.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:32:28.487 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:32:28.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:32:28.488 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:32:28.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:32:28.491 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T17:33:00.365 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7210651400640204) > weight (0.25) 2021-02-20T17:33:00.366 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T17:33:00.367 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T17:33:00.367 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T17:33:00.368 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T17:33:00.361+0000 7f6839547700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 75792) UID: 1000 2021-02-20T17:33:00.568 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:33:05.367 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T17:33:05.368 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:33:05.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:33:05.369 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T17:33:05.369 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T17:33:05.370 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T17:33:05.373 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T17:33:05.373 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T17:33:53.554 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T17:33:53.544+0000 7f7308f93700 -1 reset not still connected to 0x55aabd32b0c0 2021-02-20T17:33:53.555 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T17:33:53.544+0000 7f7308f93700 -1 reset not still connected to 0x55aac3f85190 2021-02-20T17:33:53.556 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T17:33:53.544+0000 7f7308f93700 -1 reset not still connected to 0x55aacd8bec30 2021-02-20T17:34:04.492 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8117081244290411) > weight (0.25) 2021-02-20T17:34:04.493 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.37292751258759227) > weight (0.25) 2021-02-20T17:34:04.493 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8526649671341494) > weight (0.25) 2021-02-20T17:34:04.493 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.36063310179086616) > weight (0.25) 2021-02-20T17:34:04.494 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T17:34:47.374 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7281034026899226) > weight (0.25) 2021-02-20T17:34:47.382 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T17:34:47.383 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T17:34:47.383 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T17:34:47.384 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T17:34:47.379+0000 7f8069d23700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 76166) UID: 1000 2021-02-20T17:34:47.584 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:34:55.384 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T17:34:55.384 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:34:55.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:34:55.385 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T17:34:55.385 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T17:34:55.386 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T17:34:55.389 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T17:34:55.389 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T17:35:29.495 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3039931638312481) > weight (0.25) 2021-02-20T17:35:29.508 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T17:35:29.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T17:35:29.510 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T17:35:29.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T17:35:29.507+0000 7fea23081700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 75883) UID: 1000 2021-02-20T17:35:29.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:35:36.511 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T17:35:36.526 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:35:36.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:35:36.527 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T17:35:36.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T17:35:36.529 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T17:35:36.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T17:35:36.532 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T17:36:02.390 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9872326636975257) > weight (0.25) 2021-02-20T17:36:02.397 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T17:36:02.397 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T17:36:02.397 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T17:36:02.401 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T17:36:02.394+0000 7fe05d9da700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 76257) UID: 1000 2021-02-20T17:36:02.599 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:36:08.398 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T17:36:08.398 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:36:08.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:36:08.399 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T17:36:08.399 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T17:36:08.400 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T17:36:08.402 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T17:36:08.403 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T17:37:05.533 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9605129276718949) > weight (0.25) 2021-02-20T17:37:05.534 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3579710770531589) > weight (0.25) 2021-02-20T17:37:05.534 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8333493502351247) > weight (0.25) 2021-02-20T17:37:05.534 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6673374348207043) > weight (0.25) 2021-02-20T17:37:05.535 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T17:37:19.404 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5179814247936267) > weight (0.25) 2021-02-20T17:37:19.405 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4017128742523641) > weight (0.25) 2021-02-20T17:37:19.405 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9393623725353972) > weight (0.25) 2021-02-20T17:37:19.406 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9396948519442284) > weight (0.25) 2021-02-20T17:37:19.406 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T17:38:47.407 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:38:47.434 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:38:47.435 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T17:38:47.435 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:38:47.435 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:38:47.436 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T17:38:47.431+0000 7fcc48b1a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 75499) UID: 1000 2021-02-20T17:38:47.636 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:38:53.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:38:53.538 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:38:53.538 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:38:53.539 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:38:53.541 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:38:53.541 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T17:38:57.536 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:38:57.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:38:57.537 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T17:38:57.537 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:38:57.538 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:38:57.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T17:38:57.535+0000 7f2ecce20700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 76075) UID: 1000 2021-02-20T17:38:57.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:39:03.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:39:03.640 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:39:03.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:39:03.640 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:39:03.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:39:03.643 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T17:40:19.644 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.32017467436028735) > weight (0.25) 2021-02-20T17:40:19.651 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.45952402949581517) > weight (0.25) 2021-02-20T17:40:19.651 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5237518594140673) > weight (0.25) 2021-02-20T17:40:19.651 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7681155463417144) > weight (0.25) 2021-02-20T17:40:19.652 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T17:40:37.542 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9088760371298246) > weight (0.25) 2021-02-20T17:40:37.551 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2980610407539801) > weight (0.25) 2021-02-20T17:40:37.551 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8717524175142697) > weight (0.25) 2021-02-20T17:40:37.552 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3845582553997763) > weight (0.25) 2021-02-20T17:40:37.552 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T17:42:02.653 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.848996907657281) > weight (0.25) 2021-02-20T17:42:02.689 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5748307623811597) > weight (0.25) 2021-02-20T17:42:02.689 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6793159778298766) > weight (0.25) 2021-02-20T17:42:02.689 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.622778874216389) > weight (0.25) 2021-02-20T17:42:02.690 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T17:42:27.554 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5725722088094758) > weight (0.25) 2021-02-20T17:42:27.562 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6271382618407434) > weight (0.25) 2021-02-20T17:42:27.562 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7750302094649826) > weight (0.25) 2021-02-20T17:42:27.563 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T17:42:27.564 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T17:42:27.565 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T17:42:27.566 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T17:42:27.564+0000 7f5875f2b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 75594) UID: 1000 2021-02-20T17:42:27.765 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:42:35.566 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T17:42:35.567 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:42:35.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:42:35.568 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T17:42:35.568 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T17:42:35.568 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T17:42:35.571 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T17:42:35.571 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T17:43:38.691 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:43:38.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:43:38.692 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T17:43:38.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T17:43:38.690+0000 7ff0bbf4f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 76626) UID: 1000 2021-02-20T17:43:38.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:43:41.694 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:43:41.694 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:43:41.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:43:41.695 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:43:41.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:43:41.696 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:43:41.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:43:41.699 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T17:44:32.573 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:44:32.580 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:44:32.580 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T17:44:32.581 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T17:44:32.577+0000 7fcf7d5bf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 76535) UID: 1000 2021-02-20T17:44:32.782 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:44:34.581 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:44:34.582 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:44:34.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:44:34.583 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:44:34.583 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:44:34.583 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:44:34.586 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:44:34.586 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T17:45:30.700 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:45:30.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:45:30.712 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T17:45:30.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T17:45:30.708+0000 7f514fa6f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 76877) UID: 1000 2021-02-20T17:45:30.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:45:37.713 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:45:37.714 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:45:37.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:45:37.715 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:45:37.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:45:37.715 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:45:37.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:45:37.718 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T17:46:14.588 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6984487284211691) > weight (0.25) 2021-02-20T17:46:14.589 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.46673206054582017) > weight (0.25) 2021-02-20T17:46:14.589 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2866496662674486) > weight (0.25) 2021-02-20T17:46:14.589 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T17:46:14.590 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T17:46:14.590 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T17:46:14.591 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T17:46:14.588+0000 7fb9652ca700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 76718) UID: 1000 2021-02-20T17:46:14.792 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:46:21.591 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T17:46:21.591 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:46:21.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:46:21.592 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T17:46:21.592 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T17:46:21.593 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T17:46:21.595 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T17:46:21.596 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T17:46:56.719 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3657457491121333) > weight (0.25) 2021-02-20T17:46:56.732 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7367202907118563) > weight (0.25) 2021-02-20T17:46:56.732 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6291428157152364) > weight (0.25) 2021-02-20T17:46:56.732 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9171143301994584) > weight (0.25) 2021-02-20T17:46:56.733 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T17:48:02.734 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3061663802076098) > weight (0.25) 2021-02-20T17:48:02.746 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T17:48:02.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T17:48:02.747 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T17:48:02.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T17:48:02.742+0000 7facc15ad700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 76351) UID: 1000 2021-02-20T17:48:02.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:48:10.748 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T17:48:10.749 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:48:10.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:48:10.750 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T17:48:10.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T17:48:10.751 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T17:48:10.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T17:48:10.754 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T17:48:12.597 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:48:12.598 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:48:12.598 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T17:48:12.599 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T17:48:12.594+0000 7f5ed3934700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 76968) UID: 1000 2021-02-20T17:48:12.799 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:48:15.598 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:48:15.599 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:48:15.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:48:15.600 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:48:15.600 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:48:15.600 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:48:15.603 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:48:15.603 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T17:49:20.605 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3079165222134883) > weight (0.25) 2021-02-20T17:49:20.611 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5817071665787711) > weight (0.25) 2021-02-20T17:49:20.612 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.46001111836733377) > weight (0.25) 2021-02-20T17:49:20.612 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7138369652099116) > weight (0.25) 2021-02-20T17:49:20.612 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T17:49:53.756 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:49:53.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:49:53.765 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T17:49:53.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T17:49:53.764+0000 7fee67a94700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 77062) UID: 1000 2021-02-20T17:49:53.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:49:56.767 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:49:56.767 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:49:56.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:49:56.768 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:49:56.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:49:56.769 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:49:56.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:49:56.772 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T17:51:16.613 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:51:16.615 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:51:16.615 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T17:51:16.617 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T17:51:16.611+0000 7fea943ff700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 77336) UID: 1000 2021-02-20T17:51:16.817 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:51:18.616 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:51:18.619 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:51:18.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:51:18.620 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:51:18.620 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:51:18.621 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:51:18.623 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:51:18.624 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T17:51:36.773 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3995365522824754) > weight (0.25) 2021-02-20T17:51:36.783 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T17:51:36.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T17:51:36.785 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T17:51:36.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T17:51:36.782+0000 7f63585a3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 77245) UID: 1000 2021-02-20T17:51:36.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:51:45.785 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T17:51:45.786 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:51:45.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:51:45.787 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T17:51:45.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T17:51:45.787 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T17:51:45.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T17:51:45.791 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T17:52:19.624 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7636359167305798) > weight (0.25) 2021-02-20T17:52:19.625 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T17:52:19.626 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T17:52:19.626 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T17:52:19.627 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T17:52:19.621+0000 7f7dd0728700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 76442) UID: 1000 2021-02-20T17:52:19.827 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:52:28.627 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T17:52:28.635 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:52:28.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:52:28.636 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T17:52:28.636 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T17:52:28.637 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T17:52:28.640 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T17:52:28.640 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T17:53:18.792 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6531462038147761) > weight (0.25) 2021-02-20T17:53:18.792 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.420330556925654) > weight (0.25) 2021-02-20T17:53:18.793 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7754032815568023) > weight (0.25) 2021-02-20T17:53:18.793 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7466145782770777) > weight (0.25) 2021-02-20T17:53:18.793 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T17:54:09.641 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9217291832106221) > weight (0.25) 2021-02-20T17:54:09.647 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9576418427055684) > weight (0.25) 2021-02-20T17:54:09.647 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T17:54:09.648 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T17:54:09.648 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T17:54:09.649 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T17:54:09.648+0000 7fe9fdb1e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 74181) UID: 1000 2021-02-20T17:54:10.050 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:54:16.649 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T17:54:16.655 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:54:16.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:54:16.656 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T17:54:16.656 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T17:54:16.657 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T17:54:16.660 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T17:54:16.661 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T17:54:35.794 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5789851825877566) > weight (0.25) 2021-02-20T17:54:35.800 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.92039524692571) > weight (0.25) 2021-02-20T17:54:35.800 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8254364693587989) > weight (0.25) 2021-02-20T17:54:35.800 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.49954142045134486) > weight (0.25) 2021-02-20T17:54:35.800 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T17:56:04.662 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7531987292689613) > weight (0.25) 2021-02-20T17:56:04.663 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3607011909562017) > weight (0.25) 2021-02-20T17:56:04.663 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5428291257455837) > weight (0.25) 2021-02-20T17:56:04.664 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8846923886060902) > weight (0.25) 2021-02-20T17:56:04.664 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T17:56:18.801 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2915661664088889) > weight (0.25) 2021-02-20T17:56:18.802 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.48055441513563923) > weight (0.25) 2021-02-20T17:56:18.802 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4949201156229017) > weight (0.25) 2021-02-20T17:56:18.802 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.49727359756175415) > weight (0.25) 2021-02-20T17:56:18.802 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T17:57:24.804 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7610296058914989) > weight (0.25) 2021-02-20T17:57:24.805 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9852427585464167) > weight (0.25) 2021-02-20T17:57:24.805 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6303018041265771) > weight (0.25) 2021-02-20T17:57:24.805 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9233932718603083) > weight (0.25) 2021-02-20T17:57:24.805 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T17:57:56.665 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8335470173023037) > weight (0.25) 2021-02-20T17:57:56.665 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3363179261750695) > weight (0.25) 2021-02-20T17:57:56.665 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.49045317905287344) > weight (0.25) 2021-02-20T17:57:56.666 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6414032933101613) > weight (0.25) 2021-02-20T17:57:56.666 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T17:58:26.807 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.30785519709179177) > weight (0.25) 2021-02-20T17:58:26.807 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T17:58:26.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T17:58:26.809 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T17:58:26.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T17:58:26.807+0000 7f590e31e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 77610) UID: 1000 2021-02-20T17:58:27.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:58:35.810 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T17:58:35.810 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:58:35.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:58:35.812 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T17:58:35.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T17:58:35.813 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T17:58:35.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T17:58:35.817 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T17:59:35.667 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8070611134849679) > weight (0.25) 2021-02-20T17:59:35.668 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5674659731904871) > weight (0.25) 2021-02-20T17:59:35.668 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5304530322709554) > weight (0.25) 2021-02-20T17:59:35.669 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7469022281146688) > weight (0.25) 2021-02-20T17:59:35.669 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T18:00:15.819 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8071622139583812) > weight (0.25) 2021-02-20T18:00:15.820 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8709561466945841) > weight (0.25) 2021-02-20T18:00:15.820 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7125935364174087) > weight (0.25) 2021-02-20T18:00:15.820 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T18:00:15.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T18:00:15.822 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:00:15.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T18:00:15.817+0000 7fa86f2d4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 75984) UID: 1000 2021-02-20T18:00:16.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:00:22.823 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T18:00:22.824 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:00:22.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:00:22.825 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T18:00:22.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T18:00:22.826 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T18:00:22.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T18:00:22.829 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T18:00:59.540 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T18:00:59.533+0000 7f7308f93700 -1 reset not still connected to 0x55aac7f89260 2021-02-20T18:01:22.830 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8951023857008419) > weight (0.25) 2021-02-20T18:01:22.838 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4330613310228598) > weight (0.25) 2021-02-20T18:01:22.839 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7278271822804447) > weight (0.25) 2021-02-20T18:01:22.839 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8659605823490635) > weight (0.25) 2021-02-20T18:01:22.839 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T18:01:26.670 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8360746121574362) > weight (0.25) 2021-02-20T18:01:26.670 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.26323559433444965) > weight (0.25) 2021-02-20T18:01:26.671 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T18:01:26.671 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T18:01:26.673 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T18:01:26.674 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T18:01:26.668+0000 7f5b1d8d6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 77792) UID: 1000 2021-02-20T18:01:26.873 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:01:28.675 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T18:01:28.675 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:01:28.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:01:28.676 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T18:01:28.676 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T18:01:28.676 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T18:01:28.679 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T18:01:28.679 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T18:02:58.680 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.910360874975868) > weight (0.25) 2021-02-20T18:02:58.681 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T18:02:58.682 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T18:02:58.682 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:02:58.683 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T18:02:58.679+0000 7fbde1ed8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 77701) UID: 1000 2021-02-20T18:02:58.883 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:03:05.684 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T18:03:05.684 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:03:05.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:03:05.685 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T18:03:05.685 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T18:03:05.686 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T18:03:05.689 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T18:03:05.689 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T18:03:09.841 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7097685903897593) > weight (0.25) 2021-02-20T18:03:10.102 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7210689664190788) > weight (0.25) 2021-02-20T18:03:10.103 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5204836350208164) > weight (0.25) 2021-02-20T18:03:10.103 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.46446188274051237) > weight (0.25) 2021-02-20T18:03:10.103 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T18:04:20.104 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8783621525119288) > weight (0.25) 2021-02-20T18:04:20.105 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6601337090236248) > weight (0.25) 2021-02-20T18:04:20.105 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7791228014496648) > weight (0.25) 2021-02-20T18:04:20.105 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T18:04:20.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T18:04:20.107 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T18:04:20.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T18:04:20.101+0000 7f6b02a46700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 77980) UID: 1000 2021-02-20T18:04:20.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:04:23.108 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T18:04:23.108 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:04:23.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:04:23.109 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T18:04:23.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T18:04:23.110 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T18:04:23.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T18:04:23.113 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T18:04:37.690 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.456205733040429) > weight (0.25) 2021-02-20T18:04:37.691 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T18:04:37.692 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T18:04:37.692 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T18:04:37.693 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T18:04:37.689+0000 7f73672d5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 78163) UID: 1000 2021-02-20T18:04:37.893 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:04:45.693 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T18:04:45.693 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:04:45.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:04:45.694 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T18:04:45.694 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T18:04:45.695 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T18:04:45.697 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T18:04:45.698 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T18:05:25.114 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4689670430694469) > weight (0.25) 2021-02-20T18:05:25.115 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6197838536677797) > weight (0.25) 2021-02-20T18:05:25.115 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.30696404711710323) > weight (0.25) 2021-02-20T18:05:25.115 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6597970387323402) > weight (0.25) 2021-02-20T18:05:25.115 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T18:06:25.699 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.47682959469484654) > weight (0.25) 2021-02-20T18:06:25.700 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8650487751848452) > weight (0.25) 2021-02-20T18:06:25.700 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.980079600840506) > weight (0.25) 2021-02-20T18:06:25.700 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6116737499698244) > weight (0.25) 2021-02-20T18:06:25.700 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T18:07:06.116 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:07:06.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:07:06.118 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T18:07:06.118 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:07:06.119 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:07:06.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T18:07:06.114+0000 7f00f5d89700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 77427) UID: 1000 2021-02-20T18:07:06.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:07:12.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:07:12.221 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:07:12.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:07:12.221 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:07:12.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:07:12.225 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T18:08:18.702 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.47990357604084244) > weight (0.25) 2021-02-20T18:08:18.702 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7279447180784754) > weight (0.25) 2021-02-20T18:08:18.703 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.512524243755812) > weight (0.25) 2021-02-20T18:08:18.703 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.33712886473203973) > weight (0.25) 2021-02-20T18:08:18.703 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T18:08:56.226 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:08:56.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:08:56.228 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T18:08:56.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T18:08:56.225+0000 7f36daf90700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 78457) UID: 1000 2021-02-20T18:08:56.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:08:59.229 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:08:59.230 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:08:59.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:08:59.231 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:08:59.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:08:59.231 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:08:59.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:08:59.234 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T18:10:14.235 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5643718133805573) > weight (0.25) 2021-02-20T18:10:14.236 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.34111585591490023) > weight (0.25) 2021-02-20T18:10:14.236 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6299420348759697) > weight (0.25) 2021-02-20T18:10:14.237 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.35378333479582946) > weight (0.25) 2021-02-20T18:10:14.237 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T18:10:15.704 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:10:15.706 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:10:15.706 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T18:10:15.707 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T18:10:15.703+0000 7f2725e5f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 77518) UID: 1000 2021-02-20T18:10:16.108 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:10:16.707 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:10:16.708 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:10:16.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:10:16.709 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:10:16.709 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:10:16.709 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:10:16.712 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:10:16.713 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T18:11:32.238 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7327413483216465) > weight (0.25) 2021-02-20T18:11:32.238 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8468627699377135) > weight (0.25) 2021-02-20T18:11:32.239 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.30896299018390383) > weight (0.25) 2021-02-20T18:11:32.239 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T18:11:32.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T18:11:32.240 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:11:32.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T18:11:32.234+0000 7fe700346700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 78254) UID: 1000 2021-02-20T18:11:32.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:11:39.241 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T18:11:39.241 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:11:39.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:11:39.242 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T18:11:39.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T18:11:39.243 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T18:11:39.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T18:11:39.246 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T18:11:49.714 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:11:49.716 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:11:49.716 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:11:49.717 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T18:11:49.714+0000 7f4a3bf56700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 78643) UID: 1000 2021-02-20T18:11:49.918 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:11:56.717 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:11:56.717 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:11:56.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:11:56.718 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:11:56.718 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:11:56.719 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:11:56.721 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:11:56.721 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T18:13:08.247 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2639062047424655) > weight (0.25) 2021-02-20T18:13:08.248 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5346322704556048) > weight (0.25) 2021-02-20T18:13:08.248 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7420889907355586) > weight (0.25) 2021-02-20T18:13:08.249 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5440420317436341) > weight (0.25) 2021-02-20T18:13:08.249 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T18:13:12.722 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7024854361060275) > weight (0.25) 2021-02-20T18:13:12.723 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.456981130185272) > weight (0.25) 2021-02-20T18:13:12.723 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9307108910341879) > weight (0.25) 2021-02-20T18:13:12.723 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.575396634014894) > weight (0.25) 2021-02-20T18:13:12.724 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T18:14:34.725 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:14:34.727 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:14:34.728 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T18:14:34.729 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T18:14:34.723+0000 7ff3d1a9c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 78826) UID: 1000 2021-02-20T18:14:34.929 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:14:43.729 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:14:43.729 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:14:43.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:14:43.730 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:14:43.731 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:14:43.731 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:14:43.734 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:14:43.735 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T18:14:52.250 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7555119994232204) > weight (0.25) 2021-02-20T18:14:52.250 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9333857488683063) > weight (0.25) 2021-02-20T18:14:52.251 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7832751827372374) > weight (0.25) 2021-02-20T18:14:52.251 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7833674106724392) > weight (0.25) 2021-02-20T18:14:52.251 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T18:16:40.736 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.974189367293581) > weight (0.25) 2021-02-20T18:16:40.737 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6435986146259097) > weight (0.25) 2021-02-20T18:16:40.737 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5234095589369733) > weight (0.25) 2021-02-20T18:16:40.738 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.74217294269606) > weight (0.25) 2021-02-20T18:16:40.738 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T18:16:46.252 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.25297486655786894) > weight (0.25) 2021-02-20T18:16:46.252 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T18:16:46.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T18:16:46.254 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T18:16:46.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T18:16:46.249+0000 7f4b9f7f4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 77887) UID: 1000 2021-02-20T18:16:46.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:16:49.255 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T18:16:49.256 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:16:49.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:16:49.257 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T18:16:49.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T18:16:49.257 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T18:16:49.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T18:16:49.261 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T18:17:55.740 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.514797503276502) > weight (0.25) 2021-02-20T18:17:55.741 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.27696331605124735) > weight (0.25) 2021-02-20T18:17:55.741 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T18:17:55.742 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T18:17:55.742 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T18:17:55.743 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T18:17:55.739+0000 7effa3a41700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 78071) UID: 1000 2021-02-20T18:17:56.144 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:18:03.743 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T18:18:03.743 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:18:03.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:18:03.744 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T18:18:03.744 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T18:18:03.745 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T18:18:03.748 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T18:18:03.748 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T18:18:42.262 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8484074581661898) > weight (0.25) 2021-02-20T18:18:42.262 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2753746188850289) > weight (0.25) 2021-02-20T18:18:42.263 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7851218482326655) > weight (0.25) 2021-02-20T18:18:42.263 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.25082619365150594) > weight (0.25) 2021-02-20T18:18:42.263 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T18:19:41.750 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2786249509872415) > weight (0.25) 2021-02-20T18:19:41.750 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T18:19:41.751 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T18:19:41.752 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:19:41.752 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T18:19:41.746+0000 7f87d86bc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 78345) UID: 1000 2021-02-20T18:19:42.153 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:19:45.994 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T18:19:45.989+0000 7f7b50259700 -1 rbd::mirror::ImageReplayer: 0x564bb007b680 [3/3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd] start: already running 2021-02-20T18:19:48.752 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T18:19:48.753 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:19:48.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:19:48.754 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T18:19:48.754 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T18:19:48.755 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T18:19:48.762 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T18:19:48.762 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T18:20:34.265 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:20:34.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:20:34.266 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:20:34.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T18:20:34.265+0000 7f6aaf059700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 78551) UID: 1000 2021-02-20T18:20:34.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:20:41.267 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:20:41.268 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:20:41.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:20:41.269 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:20:41.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:20:41.269 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:20:41.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:20:41.272 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T18:21:09.763 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:21:09.764 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:21:09.764 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T18:21:09.765 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T18:21:09.760+0000 7fb55cacb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 78917) UID: 1000 2021-02-20T18:21:09.965 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:21:17.766 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:21:17.766 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:21:17.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:21:17.767 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:21:17.767 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:21:17.768 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:21:17.771 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:21:17.772 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T18:22:13.273 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.29885217710665) > weight (0.25) 2021-02-20T18:22:13.274 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T18:22:13.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T18:22:13.275 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T18:22:13.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T18:22:13.271+0000 7f07fbeef700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 79011) UID: 1000 2021-02-20T18:22:13.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:22:19.275 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T18:22:19.324 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:22:19.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:22:19.325 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T18:22:19.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T18:22:19.325 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T18:22:19.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T18:22:19.328 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T18:22:28.773 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:22:28.774 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:22:28.774 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T18:22:28.775 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T18:22:28.771+0000 7fb977b52700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 79379) UID: 1000 2021-02-20T18:22:28.975 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:22:34.775 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:22:34.776 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:22:34.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:22:34.776 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:22:34.777 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:22:34.777 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:22:34.780 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:22:34.780 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T18:23:31.329 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4647090559110415) > weight (0.25) 2021-02-20T18:23:31.330 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T18:23:31.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T18:23:31.331 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T18:23:31.331 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T18:23:31.332 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:23:31.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T18:23:31.330+0000 7f71a17bc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 79470) UID: 1000 2021-02-20T18:23:31.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:23:37.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:23:37.442 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T18:23:37.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T18:23:37.444 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T18:23:37.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T18:23:37.447 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T18:24:02.781 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8490227749649025) > weight (0.25) 2021-02-20T18:24:02.782 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9142247747079718) > weight (0.25) 2021-02-20T18:24:02.782 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6389572591627514) > weight (0.25) 2021-02-20T18:24:02.782 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4569085324814802) > weight (0.25) 2021-02-20T18:24:02.783 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T18:25:27.448 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.36696356097637006) > weight (0.25) 2021-02-20T18:25:27.449 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4592522707347091) > weight (0.25) 2021-02-20T18:25:27.449 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T18:25:27.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T18:25:27.450 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T18:25:27.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T18:25:27.448+0000 7efefbdb2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 75685) UID: 1000 2021-02-20T18:25:27.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:25:32.451 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T18:25:32.452 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:25:32.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:25:32.452 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T18:25:32.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T18:25:32.453 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T18:25:32.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T18:25:32.456 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T18:25:57.784 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9530810872693243) > weight (0.25) 2021-02-20T18:25:57.785 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5978673794486649) > weight (0.25) 2021-02-20T18:25:57.785 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4969213225974679) > weight (0.25) 2021-02-20T18:25:57.785 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8991302787734557) > weight (0.25) 2021-02-20T18:25:57.785 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T18:26:53.458 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7740376742793406) > weight (0.25) 2021-02-20T18:26:53.459 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T18:26:53.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T18:26:53.460 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:26:53.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T18:26:53.454+0000 7f8a31be7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 79652) UID: 1000 2021-02-20T18:26:53.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:27:00.461 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T18:27:00.461 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:27:00.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:27:00.462 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T18:27:00.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T18:27:00.463 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T18:27:00.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T18:27:00.468 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T18:27:18.786 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5376954765505702) > weight (0.25) 2021-02-20T18:27:18.787 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6480588857771291) > weight (0.25) 2021-02-20T18:27:18.787 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8827464076927345) > weight (0.25) 2021-02-20T18:27:18.787 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T18:27:18.788 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T18:27:18.789 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:27:18.789 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T18:27:18.782+0000 7f7b50a5a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 77153) UID: 1000 2021-02-20T18:27:18.882 INFO:tasks.ceph.cluster2.osd.1.smithi090.stderr:2021-02-20T18:27:18.874+0000 7f38efdb0700 -1 reset not still connected to 0x56498340c560 2021-02-20T18:27:19.190 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:27:25.790 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T18:27:25.790 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:27:25.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:27:25.791 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T18:27:25.791 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T18:27:25.792 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T18:27:25.794 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T18:27:25.795 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T18:28:05.470 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6155992801916781) > weight (0.25) 2021-02-20T18:28:05.470 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2988899311903389) > weight (0.25) 2021-02-20T18:28:05.470 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T18:28:05.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T18:28:05.472 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T18:28:05.472 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T18:28:05.472 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:28:05.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T18:28:05.465+0000 7f7cd00ce700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 79754) UID: 1000 2021-02-20T18:28:05.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:28:11.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:28:11.575 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T18:28:11.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T18:28:11.575 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T18:28:11.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T18:28:11.579 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T18:28:43.796 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.31844933405439124) > weight (0.25) 2021-02-20T18:28:43.796 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4406347390338351) > weight (0.25) 2021-02-20T18:28:43.796 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.39800973022940445) > weight (0.25) 2021-02-20T18:28:43.797 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9989106567284811) > weight (0.25) 2021-02-20T18:28:43.797 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T18:29:11.580 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3308075192803994) > weight (0.25) 2021-02-20T18:29:11.588 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4934500391562616) > weight (0.25) 2021-02-20T18:29:11.588 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6554663695933801) > weight (0.25) 2021-02-20T18:29:11.589 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8690382291785097) > weight (0.25) 2021-02-20T18:29:11.589 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T18:30:26.798 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3952471480371912) > weight (0.25) 2021-02-20T18:30:26.810 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2596396084597258) > weight (0.25) 2021-02-20T18:30:26.810 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T18:30:26.811 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T18:30:26.811 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T18:30:26.812 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T18:30:26.806+0000 7f1a632e0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 79105) UID: 1000 2021-02-20T18:30:27.012 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:30:31.812 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T18:30:31.812 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:30:31.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:30:31.813 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T18:30:31.813 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T18:30:31.814 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T18:30:31.816 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T18:30:31.817 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T18:31:05.590 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:31:05.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:31:05.591 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T18:31:05.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T18:31:05.586+0000 7fd32e4ae700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 79288) UID: 1000 2021-02-20T18:31:05.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:31:10.592 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:31:10.593 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:31:10.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:31:10.594 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:31:10.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:31:10.594 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:31:10.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:31:10.597 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T18:31:43.818 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.415943657253244) > weight (0.25) 2021-02-20T18:31:43.819 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4615593162183057) > weight (0.25) 2021-02-20T18:31:43.819 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5861984418921578) > weight (0.25) 2021-02-20T18:31:43.819 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4391980035196448) > weight (0.25) 2021-02-20T18:31:43.819 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T18:32:34.599 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4921980898453635) > weight (0.25) 2021-02-20T18:32:34.613 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6569747217824043) > weight (0.25) 2021-02-20T18:32:34.613 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9964002358271445) > weight (0.25) 2021-02-20T18:32:34.613 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.2592847500912002) > weight (0.25) 2021-02-20T18:32:34.614 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T18:33:29.820 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:33:29.821 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:33:29.822 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T18:33:29.822 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:33:29.822 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:33:29.823 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T18:33:29.819+0000 7ff5df243700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 79561) UID: 1000 2021-02-20T18:33:30.023 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:33:35.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:33:35.924 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:33:35.925 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:33:35.925 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:33:35.928 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:33:35.928 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T18:33:42.615 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:33:42.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:33:42.616 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:33:42.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T18:33:42.611+0000 7febd9044700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 80211) UID: 1000 2021-02-20T18:33:42.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:33:49.616 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:33:49.617 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:33:49.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:33:49.618 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:33:49.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:33:49.618 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:33:49.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:33:49.621 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T18:34:52.930 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9087289590488663) > weight (0.25) 2021-02-20T18:34:52.930 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8006531094339756) > weight (0.25) 2021-02-20T18:34:52.931 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T18:34:52.931 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T18:34:52.932 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T18:34:52.933 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T18:34:52.926+0000 7fc7aacf6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 80120) UID: 1000 2021-02-20T18:34:52.940 INFO:tasks.ceph.cluster2.osd.0.smithi090.stderr:2021-02-20T18:34:52.934+0000 7f7bd2757700 -1 reset not still connected to 0x5624d64176c0 2021-02-20T18:34:53.133 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:34:55.933 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T18:34:55.933 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:34:55.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:34:55.934 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T18:34:55.934 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T18:34:55.935 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T18:34:55.937 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T18:34:55.937 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T18:35:22.623 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3931084073954033) > weight (0.25) 2021-02-20T18:35:22.623 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7456955298133557) > weight (0.25) 2021-02-20T18:35:22.624 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3575514376161596) > weight (0.25) 2021-02-20T18:35:22.624 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9944617821029914) > weight (0.25) 2021-02-20T18:35:22.624 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T18:36:21.938 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3429009582750049) > weight (0.25) 2021-02-20T18:36:21.939 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8143659544265448) > weight (0.25) 2021-02-20T18:36:21.939 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5978703545766526) > weight (0.25) 2021-02-20T18:36:21.939 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4185520152080492) > weight (0.25) 2021-02-20T18:36:21.940 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T18:36:43.625 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6387057946582028) > weight (0.25) 2021-02-20T18:36:43.626 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T18:36:43.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T18:36:43.627 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T18:36:43.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T18:36:43.624+0000 7fd854db3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 79845) UID: 1000 2021-02-20T18:36:43.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:36:46.628 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T18:36:46.628 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:36:46.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:36:46.629 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T18:36:46.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T18:36:46.629 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T18:36:46.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T18:36:46.632 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T18:37:19.037 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T18:37:19.029+0000 7fb533c45700 -1 reset not still connected to 0x558b94b8f5f0 2021-02-20T18:37:30.941 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:37:30.949 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:37:30.950 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T18:37:30.951 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T18:37:30.943+0000 7fb28ae81700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 80302) UID: 1000 2021-02-20T18:37:31.150 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:37:36.951 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:37:36.952 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:37:36.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:37:36.952 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:37:36.953 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:37:36.953 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:37:36.957 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:37:36.957 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T18:38:15.633 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7497392442119393) > weight (0.25) 2021-02-20T18:38:15.640 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8291286142375719) > weight (0.25) 2021-02-20T18:38:15.640 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7585371958346782) > weight (0.25) 2021-02-20T18:38:15.640 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.858225216716447) > weight (0.25) 2021-02-20T18:38:15.641 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T18:39:22.958 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5845742654802846) > weight (0.25) 2021-02-20T18:39:22.966 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.46622752877850504) > weight (0.25) 2021-02-20T18:39:22.967 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8641273536470898) > weight (0.25) 2021-02-20T18:39:22.967 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T18:39:22.968 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T18:39:22.969 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T18:39:22.970 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T18:39:22.961+0000 7fa39cdfc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 79936) UID: 1000 2021-02-20T18:39:23.169 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:39:25.642 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.31074628766185497) > weight (0.25) 2021-02-20T18:39:25.642 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T18:39:25.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T18:39:25.643 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T18:39:25.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T18:39:25.637+0000 7f3cdf406700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 80580) UID: 1000 2021-02-20T18:39:25.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:39:28.969 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T18:39:28.970 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:39:28.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:39:28.971 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T18:39:28.972 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T18:39:28.972 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T18:39:28.975 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T18:39:28.976 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T18:39:31.644 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T18:39:31.645 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:39:31.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:39:31.646 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T18:39:31.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T18:39:31.647 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T18:39:31.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T18:39:31.651 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T18:40:46.653 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.596752110664645) > weight (0.25) 2021-02-20T18:40:46.653 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7155971981970513) > weight (0.25) 2021-02-20T18:40:46.654 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9651420128239679) > weight (0.25) 2021-02-20T18:40:46.654 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6268744162123197) > weight (0.25) 2021-02-20T18:40:46.654 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T18:41:09.977 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:41:09.978 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:41:09.978 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T18:41:09.979 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T18:41:09.971+0000 7f986e734700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 80671) UID: 1000 2021-02-20T18:41:10.180 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:41:15.979 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:41:15.980 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:41:15.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:41:15.981 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:41:15.982 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:41:15.982 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:41:15.986 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:41:15.987 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T18:42:18.987 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9018749490833202) > weight (0.25) 2021-02-20T18:42:18.988 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9104019656936212) > weight (0.25) 2021-02-20T18:42:18.988 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.867087222568751) > weight (0.25) 2021-02-20T18:42:18.989 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7996493447445883) > weight (0.25) 2021-02-20T18:42:18.989 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T18:42:43.655 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:42:43.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:42:43.663 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:42:43.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T18:42:43.658+0000 7fbc6e5d9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 80394) UID: 1000 2021-02-20T18:42:43.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:42:50.665 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:42:50.665 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:42:50.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:42:50.666 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:42:50.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:42:50.667 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:42:50.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:42:50.669 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T18:43:18.990 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5403877680126513) > weight (0.25) 2021-02-20T18:43:18.991 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8638903804579742) > weight (0.25) 2021-02-20T18:43:18.991 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7266959497234333) > weight (0.25) 2021-02-20T18:43:18.991 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T18:43:18.992 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T18:43:18.992 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T18:43:18.993 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T18:43:18.985+0000 7f604b02b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 80763) UID: 1000 2021-02-20T18:43:19.194 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:43:27.994 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T18:43:28.002 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:43:28.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:43:28.007 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T18:43:28.008 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T18:43:28.009 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T18:43:28.012 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T18:43:28.012 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T18:44:37.670 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:44:37.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:44:37.680 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T18:44:37.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T18:44:37.676+0000 7f1e695f1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 81036) UID: 1000 2021-02-20T18:44:37.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:44:45.681 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:44:45.682 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:44:45.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:44:45.683 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:44:45.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:44:45.683 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:44:45.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:44:45.686 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T18:45:10.014 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7382887322642113) > weight (0.25) 2021-02-20T18:45:10.022 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.37650646756230144) > weight (0.25) 2021-02-20T18:45:10.022 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5157352848295641) > weight (0.25) 2021-02-20T18:45:10.022 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7981814539639731) > weight (0.25) 2021-02-20T18:45:10.023 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T18:45:51.687 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8127200664761044) > weight (0.25) 2021-02-20T18:45:51.698 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8407817580471421) > weight (0.25) 2021-02-20T18:45:51.698 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7792277658592679) > weight (0.25) 2021-02-20T18:45:51.699 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5669487243016469) > weight (0.25) 2021-02-20T18:45:51.699 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T18:46:36.024 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.817318209455371) > weight (0.25) 2021-02-20T18:46:36.025 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.957761396902476) > weight (0.25) 2021-02-20T18:46:36.025 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5300547640360982) > weight (0.25) 2021-02-20T18:46:36.025 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9028896796085479) > weight (0.25) 2021-02-20T18:46:36.026 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T18:47:36.699 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9826539776840707) > weight (0.25) 2021-02-20T18:47:36.700 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9430806006362207) > weight (0.25) 2021-02-20T18:47:36.700 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6200082622182761) > weight (0.25) 2021-02-20T18:47:36.701 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T18:47:36.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T18:47:36.702 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T18:47:36.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T18:47:36.693+0000 7f716d1fe700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 78735) UID: 1000 2021-02-20T18:47:37.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:47:38.703 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T18:47:38.704 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:47:38.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:47:38.705 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T18:47:38.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T18:47:38.706 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T18:47:38.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T18:47:38.709 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T18:47:58.027 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8332136140304124) > weight (0.25) 2021-02-20T18:47:58.027 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8431044944082399) > weight (0.25) 2021-02-20T18:47:58.028 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5335542469459263) > weight (0.25) 2021-02-20T18:47:58.028 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.950630899891806) > weight (0.25) 2021-02-20T18:47:58.029 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T18:49:33.710 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7650855183639631) > weight (0.25) 2021-02-20T18:49:33.711 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.40179268230521636) > weight (0.25) 2021-02-20T18:49:33.711 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6764667105573041) > weight (0.25) 2021-02-20T18:49:33.711 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T18:49:33.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T18:49:33.713 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T18:49:33.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T18:49:33.707+0000 7f9fa7aa8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 81312) UID: 1000 2021-02-20T18:49:33.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:49:35.713 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T18:49:35.714 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:49:35.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:49:35.715 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T18:49:35.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T18:49:35.716 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T18:49:35.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T18:49:35.719 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T18:49:39.030 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:49:39.031 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:49:39.031 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:49:39.035 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T18:49:39.027+0000 7fceaee92700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 80945) UID: 1000 2021-02-20T18:49:39.235 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:49:46.032 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:49:46.032 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:49:46.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:49:46.033 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:49:46.033 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:49:46.033 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:49:46.036 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:49:46.037 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T18:50:56.719 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8940345494814614) > weight (0.25) 2021-02-20T18:50:56.720 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9921052408572106) > weight (0.25) 2021-02-20T18:50:56.720 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.693828232678504) > weight (0.25) 2021-02-20T18:50:56.721 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T18:50:56.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T18:50:56.722 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T18:50:56.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T18:50:56.713+0000 7f81e0195700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 81411) UID: 1000 2021-02-20T18:50:56.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:51:01.723 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T18:51:01.724 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:51:01.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:51:01.724 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T18:51:01.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T18:51:01.725 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T18:51:01.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T18:51:01.728 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T18:51:18.038 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5812219732817635) > weight (0.25) 2021-02-20T18:51:18.038 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9671662828520202) > weight (0.25) 2021-02-20T18:51:18.039 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7099368186257317) > weight (0.25) 2021-02-20T18:51:18.039 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T18:51:18.039 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T18:51:18.040 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T18:51:18.040 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T18:51:18.033+0000 7f8a12cd7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 81127) UID: 1000 2021-02-20T18:51:18.241 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:51:19.026 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T18:51:19.017+0000 7f7308f93700 -1 reset not still connected to 0x55aab71de410 2021-02-20T18:51:19.027 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T18:51:19.017+0000 7f7308f93700 -1 reset not still connected to 0x55aab71e02f0 2021-02-20T18:51:19.028 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T18:51:19.017+0000 7f7308f93700 -1 reset not still connected to 0x55aac2359740 2021-02-20T18:51:21.041 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T18:51:21.041 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:51:21.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:51:21.042 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T18:51:21.042 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T18:51:21.042 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T18:51:21.045 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T18:51:21.045 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T18:52:34.046 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8886778767741966) > weight (0.25) 2021-02-20T18:52:34.062 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8406908068974176) > weight (0.25) 2021-02-20T18:52:34.063 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9265360163636985) > weight (0.25) 2021-02-20T18:52:34.063 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T18:52:34.064 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T18:52:34.064 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T18:52:34.065 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T18:52:34.060+0000 7f3104b01700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 81685) UID: 1000 2021-02-20T18:52:34.265 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:52:42.065 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T18:52:42.066 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:52:42.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:52:42.067 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T18:52:42.067 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T18:52:42.068 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T18:52:42.071 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T18:52:42.071 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T18:52:54.729 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6184333067020562) > weight (0.25) 2021-02-20T18:52:54.730 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5061172858489873) > weight (0.25) 2021-02-20T18:52:54.730 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T18:52:54.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T18:52:54.731 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T18:52:54.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T18:52:54.727+0000 7f98b3618700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 80027) UID: 1000 2021-02-20T18:52:54.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:52:56.731 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T18:52:56.732 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:52:56.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:52:56.733 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T18:52:56.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T18:52:56.733 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T18:52:56.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T18:52:56.736 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T18:53:19.037 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T18:53:19.029+0000 7fb533c45700 -1 reset not still connected to 0x558b94b8f5f0 2021-02-20T18:54:28.738 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7283788286473637) > weight (0.25) 2021-02-20T18:54:28.738 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9749200316295576) > weight (0.25) 2021-02-20T18:54:28.739 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T18:54:28.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T18:54:28.740 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T18:54:28.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T18:54:28.734+0000 7fa802899700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 81868) UID: 1000 2021-02-20T18:54:28.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:54:30.072 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:54:30.072 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:54:30.073 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:54:30.073 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T18:54:30.066+0000 7f87d547d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 81502) UID: 1000 2021-02-20T18:54:30.274 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:54:36.740 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T18:54:36.741 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:54:36.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:54:36.742 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T18:54:36.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T18:54:36.743 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T18:54:36.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T18:54:36.746 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T18:54:37.073 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:54:37.074 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:54:37.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:54:37.074 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:54:37.075 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:54:37.075 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:54:37.078 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:54:37.078 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T18:56:00.079 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7227138120592881) > weight (0.25) 2021-02-20T18:56:00.080 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T18:56:00.081 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T18:56:00.082 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:56:00.083 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T18:56:00.076+0000 7f933961d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 79196) UID: 1000 2021-02-20T18:56:00.484 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:56:03.747 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.40246315325832926) > weight (0.25) 2021-02-20T18:56:03.747 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9235531425908734) > weight (0.25) 2021-02-20T18:56:03.747 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T18:56:03.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T18:56:03.748 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T18:56:03.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T18:56:03.744+0000 7fdb11ee5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 81959) UID: 1000 2021-02-20T18:56:03.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:56:07.083 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T18:56:07.084 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:56:07.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:56:07.084 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T18:56:07.085 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T18:56:07.085 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T18:56:07.088 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T18:56:07.088 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T18:56:11.749 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T18:56:11.750 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:56:11.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:56:11.751 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T18:56:11.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T18:56:11.751 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T18:56:11.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T18:56:11.754 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T18:57:02.257 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T18:57:02.111+0000 7f7308f93700 -1 reset not still connected to 0x55aabca9abe0 2021-02-20T18:57:23.755 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9050573440832413) > weight (0.25) 2021-02-20T18:57:23.756 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.46238062706897154) > weight (0.25) 2021-02-20T18:57:23.756 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4012430633648195) > weight (0.25) 2021-02-20T18:57:23.756 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.40753639509684214) > weight (0.25) 2021-02-20T18:57:23.756 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T18:57:59.089 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8520543750396283) > weight (0.25) 2021-02-20T18:57:59.090 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.38770677524826613) > weight (0.25) 2021-02-20T18:57:59.090 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T18:57:59.091 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T18:57:59.091 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T18:57:59.095 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T18:57:59.090+0000 7fb8fd5b5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 80485) UID: 1000 2021-02-20T18:57:59.495 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:58:03.092 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T18:58:03.096 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:58:03.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:58:03.097 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T18:58:03.097 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T18:58:03.097 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T18:58:03.100 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T18:58:03.100 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T18:58:46.758 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3899310778920748) > weight (0.25) 2021-02-20T18:58:46.761 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T18:58:46.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T18:58:46.762 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T18:58:46.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T18:58:46.757+0000 7f8a2199c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 80854) UID: 1000 2021-02-20T18:58:46.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:58:48.764 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T18:58:48.764 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:58:48.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:58:48.765 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T18:58:48.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T18:58:48.765 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T18:58:48.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T18:58:48.768 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T18:59:39.102 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9241854365203449) > weight (0.25) 2021-02-20T18:59:39.102 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T18:59:39.103 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T18:59:39.103 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T18:59:39.104 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T18:59:39.096+0000 7f9b1441e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 82144) UID: 1000 2021-02-20T18:59:39.305 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:59:43.105 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T18:59:43.131 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:59:43.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:59:43.132 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T18:59:43.133 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T18:59:43.133 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T18:59:43.141 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T18:59:43.141 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T19:00:14.770 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.91592597176637) > weight (0.25) 2021-02-20T19:00:14.770 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9028021140414872) > weight (0.25) 2021-02-20T19:00:14.770 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.944889980634253) > weight (0.25) 2021-02-20T19:00:14.771 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9976364773144234) > weight (0.25) 2021-02-20T19:00:14.771 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T19:00:59.142 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9673020677296333) > weight (0.25) 2021-02-20T19:00:59.152 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T19:00:59.153 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T19:00:59.153 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T19:00:59.154 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T19:00:59.147+0000 7f67ff488700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 82508) UID: 1000 2021-02-20T19:00:59.354 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:01:01.154 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T19:01:01.155 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:01:01.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:01:01.156 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T19:01:01.156 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T19:01:01.156 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T19:01:01.159 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T19:01:01.159 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T19:01:28.772 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.43482012870146525) > weight (0.25) 2021-02-20T19:01:28.773 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:01:28.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:01:28.774 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T19:01:28.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T19:01:28.767+0000 7f18038de700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 82417) UID: 1000 2021-02-20T19:01:28.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:01:29.775 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:01:29.775 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:01:29.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:01:29.776 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:01:29.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:01:29.777 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:01:29.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:01:29.780 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T19:02:44.160 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8203340617494838) > weight (0.25) 2021-02-20T19:02:44.185 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4524585600799248) > weight (0.25) 2021-02-20T19:02:44.185 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T19:02:44.186 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T19:02:44.186 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T19:02:44.187 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T19:02:44.181+0000 7f6ad4656700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 82326) UID: 1000 2021-02-20T19:02:44.388 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:02:49.187 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T19:02:49.188 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:02:49.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:02:49.189 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T19:02:49.189 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T19:02:49.189 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T19:02:49.193 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T19:02:49.193 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T19:03:20.781 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3702310606795025) > weight (0.25) 2021-02-20T19:03:20.782 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.28922882101834124) > weight (0.25) 2021-02-20T19:03:20.783 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9749744288332386) > weight (0.25) 2021-02-20T19:03:20.783 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.376014462725216) > weight (0.25) 2021-02-20T19:03:20.783 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T19:04:07.194 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6635286514689375) > weight (0.25) 2021-02-20T19:04:07.194 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9411310000499965) > weight (0.25) 2021-02-20T19:04:07.195 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T19:04:07.195 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T19:04:07.196 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T19:04:07.196 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T19:04:07.192+0000 7efe08fd7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 82783) UID: 1000 2021-02-20T19:04:07.397 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:04:14.197 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T19:04:14.198 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:04:14.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:04:14.198 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T19:04:14.199 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T19:04:14.199 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T19:04:14.202 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T19:04:14.202 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T19:05:13.784 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.88184141229748) > weight (0.25) 2021-02-20T19:05:13.795 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.396989814301704) > weight (0.25) 2021-02-20T19:05:13.796 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9989637721250215) > weight (0.25) 2021-02-20T19:05:13.796 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8287730377338177) > weight (0.25) 2021-02-20T19:05:13.796 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T19:06:10.204 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8498206706604708) > weight (0.25) 2021-02-20T19:06:10.211 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T19:06:10.212 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T19:06:10.213 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T19:06:10.213 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T19:06:10.206+0000 7fa494661700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 82600) UID: 1000 2021-02-20T19:06:10.414 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:06:11.214 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T19:06:11.214 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:06:11.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:06:11.215 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T19:06:11.215 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T19:06:11.216 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T19:06:11.218 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T19:06:11.219 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T19:06:15.797 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9905027828095156) > weight (0.25) 2021-02-20T19:06:15.797 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:06:15.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:06:15.798 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T19:06:15.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T19:06:15.790+0000 7f86be637700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 82692) UID: 1000 2021-02-20T19:06:15.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:06:24.799 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:06:24.800 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:06:24.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:06:24.801 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:06:24.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:06:24.801 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:06:24.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:06:24.804 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T19:07:33.219 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9027343596338469) > weight (0.25) 2021-02-20T19:07:33.220 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4408185684364472) > weight (0.25) 2021-02-20T19:07:33.220 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9973760056270491) > weight (0.25) 2021-02-20T19:07:33.221 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T19:07:33.221 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T19:07:33.222 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T19:07:33.222 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T19:07:33.212+0000 7f452c8b4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 81777) UID: 1000 2021-02-20T19:07:33.623 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:07:37.222 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T19:07:37.223 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:07:37.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:07:37.224 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T19:07:37.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T19:07:37.224 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T19:07:37.227 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T19:07:37.227 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T19:07:51.805 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8284569923547743) > weight (0.25) 2021-02-20T19:07:51.805 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:07:51.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:07:51.806 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T19:07:51.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T19:07:51.800+0000 7ff5c0fe3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 83059) UID: 1000 2021-02-20T19:07:52.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:07:58.807 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:07:58.808 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:07:58.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:07:58.808 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:07:58.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:07:58.809 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:07:58.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:07:58.812 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T19:09:06.813 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:09:06.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:09:06.814 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T19:09:06.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T19:09:06.807+0000 7fd01b39c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 81218) UID: 1000 2021-02-20T19:09:07.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:09:10.815 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:09:10.815 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:09:10.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:09:10.816 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:09:10.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:09:10.817 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:09:10.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:09:10.820 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T19:09:31.228 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5007323038553038) > weight (0.25) 2021-02-20T19:09:31.229 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T19:09:31.229 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T19:09:31.230 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T19:09:31.231 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T19:09:31.222+0000 7fbbd5958700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 82968) UID: 1000 2021-02-20T19:09:31.431 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:09:32.237 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T19:09:32.238 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:09:32.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:09:32.238 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T19:09:32.239 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T19:09:32.239 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T19:09:32.249 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T19:09:32.250 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T19:10:11.821 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8680072367436462) > weight (0.25) 2021-02-20T19:10:11.829 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4616198077303416) > weight (0.25) 2021-02-20T19:10:11.829 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4425231259314709) > weight (0.25) 2021-02-20T19:10:11.830 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.25874069520347) > weight (0.25) 2021-02-20T19:10:11.830 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T19:11:10.252 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:11:10.389 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:11:10.390 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T19:11:10.391 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:11:10.391 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:11:10.392 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T19:11:10.385+0000 7f3de1d43700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 82033) UID: 1000 2021-02-20T19:11:10.791 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:11:16.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:11:16.492 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:11:16.493 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:11:16.493 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:11:16.496 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:11:16.496 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T19:11:27.831 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7819216522984455) > weight (0.25) 2021-02-20T19:11:27.832 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5525965891494609) > weight (0.25) 2021-02-20T19:11:27.832 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.34034608516612197) > weight (0.25) 2021-02-20T19:11:27.832 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6976609801607191) > weight (0.25) 2021-02-20T19:11:27.832 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T19:13:11.833 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8539672328178867) > weight (0.25) 2021-02-20T19:13:11.834 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8417398353968143) > weight (0.25) 2021-02-20T19:13:11.834 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6428906108881406) > weight (0.25) 2021-02-20T19:13:11.835 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T19:13:11.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T19:13:11.836 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T19:13:11.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:2021-02-20T19:13:11.831+0000 7fe8d2060700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 81594) UID: 1000 2021-02-20T19:13:12.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:13:13.497 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7646851969933317) > weight (0.25) 2021-02-20T19:13:13.497 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6615401800485368) > weight (0.25) 2021-02-20T19:13:13.497 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7577098696983233) > weight (0.25) 2021-02-20T19:13:13.498 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5597560254095815) > weight (0.25) 2021-02-20T19:13:13.498 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T19:13:14.837 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T19:13:14.837 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:13:14.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:13:14.838 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T19:13:14.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T19:13:14.839 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T19:13:14.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T19:13:14.842 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T19:14:24.499 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8896921765343577) > weight (0.25) 2021-02-20T19:14:26.043 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T19:14:26.044 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T19:14:26.044 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T19:14:26.045 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6586828377142891) > weight (0.25) 2021-02-20T19:14:26.046 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6908620852526562) > weight (0.25) 2021-02-20T19:14:26.046 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2598887692778923) > weight (0.25) 2021-02-20T19:14:26.046 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6363450303134973) > weight (0.25) 2021-02-20T19:14:26.046 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T19:14:26.047 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T19:14:26.037+0000 7f41d0078700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 83432) UID: 1000 2021-02-20T19:14:26.446 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:14:30.253 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T19:14:30.245+0000 7f203867e700 -1 rbd::mirror::ImageReplayer: 0x56026a89db80 [3/3ebe21a5-0e1c-4743-9a5e-dfd801dcdefd] start: already running 2021-02-20T19:14:30.253 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T19:14:30.245+0000 7f203867e700 -1 rbd::mirror::ImageReplayer: 0x56026a89ed00 [3/5cc4eeb6-e809-410f-8db4-a0f6f871e5be] start: already running 2021-02-20T19:14:30.277 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T19:14:30.269+0000 7f2038e7f700 -1 rbd::mirror::ImageReplayer: 0x56026a89e580 [3/8d0e16c9-d3d2-4b9c-a2e9-c0ac1f36de1a] start: already running 2021-02-20T19:14:30.277 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T19:14:30.269+0000 7f2038e7f700 -1 rbd::mirror::ImageReplayer: 0x56026a89e800 [3/8ee65cf4-12ca-4ca6-b1e7-745c00784f3d] start: already running 2021-02-20T19:14:30.277 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T19:14:30.269+0000 7f2038e7f700 -1 rbd::mirror::ImageReplayer: 0x56026a89ea80 [3/9781de1c-0ffd-4a24-ae0e-180268ec6066] start: already running 2021-02-20T19:14:30.277 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T19:14:30.269+0000 7f2038e7f700 -1 rbd::mirror::ImageReplayer: 0x56026a89f480 [3/a39316e2-bbbe-40ae-9620-51f282c47f6c] start: already running 2021-02-20T19:14:30.278 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T19:14:30.269+0000 7f2038e7f700 -1 rbd::mirror::ImageReplayer: 0x56026a89f700 [3/a4f4a9dd-701f-4734-a679-4beea65adecf] start: already running 2021-02-20T19:14:30.278 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T19:14:30.269+0000 7f2038e7f700 -1 rbd::mirror::ImageReplayer: 0x56026a89f980 [3/b18825d4-ad79-43d5-95f8-bbc84838ad3c] start: already running 2021-02-20T19:14:30.278 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T19:14:30.269+0000 7f2038e7f700 -1 rbd::mirror::ImageReplayer: 0x56026a89fc00 [3/b614e8f1-3d22-46ce-9b70-32b4f254320a] start: already running 2021-02-20T19:14:30.365 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T19:14:30.353+0000 7f7308f93700 -1 reset not still connected to 0x55aabca9a080 2021-02-20T19:14:30.367 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T19:14:30.357+0000 7f7308f93700 -1 reset not still connected to 0x55aac3f84a40 2021-02-20T19:14:30.367 INFO:tasks.ceph.cluster2.osd.2.smithi090.stderr:2021-02-20T19:14:30.357+0000 7f7308f93700 -1 reset not still connected to 0x55aac7f882f0 2021-02-20T19:14:35.046 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T19:14:35.046 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:14:35.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:14:35.047 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T19:14:35.047 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T19:14:35.048 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T19:14:35.050 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T19:14:35.051 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T19:16:04.048 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.46486427365723826) > weight (0.25) 2021-02-20T19:16:04.058 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:16:04.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:16:04.060 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T19:16:04.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T19:16:04.056+0000 7fc1ffae3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 83242) UID: 1000 2021-02-20T19:16:04.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:16:12.061 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:16:12.061 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:16:12.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:16:12.062 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:16:12.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:16:12.063 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:16:12.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:16:12.066 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T19:16:14.052 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:16:14.052 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:16:14.053 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T19:16:14.054 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T19:16:14.047+0000 7f5b89671700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 83524) UID: 1000 2021-02-20T19:16:14.254 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:16:17.054 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:16:17.054 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:16:17.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:16:17.055 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:16:17.055 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:16:17.055 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:16:17.058 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:16:17.058 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T19:17:23.060 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5435636507806282) > weight (0.25) 2021-02-20T19:17:23.060 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2978670026906002) > weight (0.25) 2021-02-20T19:17:23.061 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7686683473048395) > weight (0.25) 2021-02-20T19:17:23.061 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.355761575334677) > weight (0.25) 2021-02-20T19:17:23.061 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T19:17:50.067 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8700861231688374) > weight (0.25) 2021-02-20T19:17:50.072 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2808909907346082) > weight (0.25) 2021-02-20T19:17:50.072 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7795567937336794) > weight (0.25) 2021-02-20T19:17:50.072 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5815582050055395) > weight (0.25) 2021-02-20T19:17:50.073 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T19:18:38.063 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:18:38.073 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:18:38.074 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T19:18:38.075 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T19:18:38.065+0000 7f5b98f9f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 83900) UID: 1000 2021-02-20T19:18:38.275 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:18:47.074 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:18:47.083 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:18:47.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:18:47.084 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:18:47.084 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:18:47.084 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:18:47.087 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:18:47.087 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T19:19:10.073 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.44075905866079124) > weight (0.25) 2021-02-20T19:19:10.082 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6941899482920413) > weight (0.25) 2021-02-20T19:19:10.082 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.602430970350551) > weight (0.25) 2021-02-20T19:19:10.083 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7540302407030909) > weight (0.25) 2021-02-20T19:19:10.083 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T19:20:07.088 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:20:07.092 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:20:07.093 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T19:20:07.094 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T19:20:07.087+0000 7f0eee03e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 83995) UID: 1000 2021-02-20T19:20:07.294 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:20:12.094 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:20:12.094 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:20:12.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:20:12.095 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:20:12.095 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:20:12.096 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:20:12.098 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:20:12.099 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T19:20:42.084 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.731699898475216) > weight (0.25) 2021-02-20T19:20:42.092 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:20:42.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:20:42.093 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T19:20:42.093 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:20:42.094 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:20:42.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T19:20:42.087+0000 7eff77371700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 83809) UID: 1000 2021-02-20T19:20:42.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:20:48.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:20:48.196 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:20:48.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:20:48.197 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:20:48.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:20:48.200 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T19:22:00.100 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:22:00.102 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:22:00.102 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T19:22:00.103 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T19:22:00.098+0000 7f9eca709700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 84087) UID: 1000 2021-02-20T19:22:00.303 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:22:03.103 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:22:03.104 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:22:03.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:22:03.105 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:22:03.105 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:22:03.106 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:22:03.109 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:22:03.109 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T19:22:15.203 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9118209564675881) > weight (0.25) 2021-02-20T19:22:15.205 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:22:15.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:22:15.207 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T19:22:15.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T19:22:15.201+0000 7f2ce86c6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 84179) UID: 1000 2021-02-20T19:22:15.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:22:21.209 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:22:21.210 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:22:21.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:22:21.211 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:22:21.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:22:21.212 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:22:21.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:22:21.218 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T19:23:13.110 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:23:13.119 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:23:13.119 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T19:23:13.120 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T19:23:13.112+0000 7f94252d0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 84270) UID: 1000 2021-02-20T19:23:13.320 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:23:14.120 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:23:14.121 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:23:14.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:23:14.122 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:23:14.122 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:23:14.122 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:23:14.125 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:23:14.125 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T19:23:23.219 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9728659168548943) > weight (0.25) 2021-02-20T19:23:23.219 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:23:23.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:23:23.220 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T19:23:23.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T19:23:23.216+0000 7f203f3ed700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 84361) UID: 1000 2021-02-20T19:23:23.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:23:29.221 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:23:29.221 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:23:29.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:23:29.222 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:23:29.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:23:29.223 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:23:29.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:23:29.226 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T19:24:14.126 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9441803619114137) > weight (0.25) 2021-02-20T19:24:14.135 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.29731443542742364) > weight (0.25) 2021-02-20T19:24:14.135 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5429521921362438) > weight (0.25) 2021-02-20T19:24:14.135 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6893535139409934) > weight (0.25) 2021-02-20T19:24:14.136 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T19:25:17.137 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7758521252171793) > weight (0.25) 2021-02-20T19:25:17.159 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T19:25:17.160 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T19:25:17.161 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T19:25:17.162 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T19:25:17.154+0000 7faa0a2f4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 83707) UID: 1000 2021-02-20T19:25:17.362 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:25:21.227 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.274363736325246) > weight (0.25) 2021-02-20T19:25:21.227 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:25:21.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:25:21.228 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T19:25:21.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T19:25:21.222+0000 7f50a73e0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 84543) UID: 1000 2021-02-20T19:25:21.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:25:22.161 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T19:25:22.162 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:25:22.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:25:22.163 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T19:25:22.163 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T19:25:22.163 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T19:25:22.166 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T19:25:22.167 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T19:25:25.228 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:25:25.229 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:25:25.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:25:25.230 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:25:25.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:25:25.230 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:25:25.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:25:25.243 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T19:26:29.244 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6469858377536444) > weight (0.25) 2021-02-20T19:26:29.257 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:26:29.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:26:29.258 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T19:26:29.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T19:26:29.249+0000 7fd33ce5e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 84728) UID: 1000 2021-02-20T19:26:29.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:26:32.259 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:26:32.260 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:26:32.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:26:32.260 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:26:32.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:26:32.261 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:26:32.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:26:32.264 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T19:27:16.167 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:27:16.179 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:27:16.181 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T19:27:16.181 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T19:27:16.176+0000 7fb594903700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 84452) UID: 1000 2021-02-20T19:27:16.381 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:27:24.182 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:27:24.189 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:27:24.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:27:24.190 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:27:24.190 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:27:24.192 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:27:24.197 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:27:24.197 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T19:28:17.265 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8783888028256907) > weight (0.25) 2021-02-20T19:28:17.276 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3827419865932359) > weight (0.25) 2021-02-20T19:28:17.277 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4024758804543488) > weight (0.25) 2021-02-20T19:28:17.277 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8389868886277474) > weight (0.25) 2021-02-20T19:28:17.277 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T19:28:28.198 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4254739169883517) > weight (0.25) 2021-02-20T19:28:28.205 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6298444029077911) > weight (0.25) 2021-02-20T19:28:28.205 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5729655008803946) > weight (0.25) 2021-02-20T19:28:28.205 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T19:28:28.206 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T19:28:28.206 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T19:28:28.207 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T19:28:28.199+0000 7f7042d78700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 83150) UID: 1000 2021-02-20T19:28:28.407 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:28:32.207 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T19:28:32.211 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:28:32.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:28:32.212 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T19:28:32.212 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T19:28:32.212 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T19:28:32.215 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T19:28:32.215 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T19:30:11.278 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6318050548786139) > weight (0.25) 2021-02-20T19:30:11.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5077703597851685) > weight (0.25) 2021-02-20T19:30:11.279 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T19:30:11.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T19:30:11.281 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T19:30:11.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:2021-02-20T19:30:11.273+0000 7fe68724f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 82235) UID: 1000 2021-02-20T19:30:11.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:30:18.281 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T19:30:18.282 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:30:18.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:30:18.283 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T19:30:18.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T19:30:18.283 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T19:30:18.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T19:30:18.287 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T19:30:27.217 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:30:27.218 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:30:27.218 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T19:30:27.219 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T19:30:27.209+0000 7f9896756700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 84910) UID: 1000 2021-02-20T19:30:27.420 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:30:35.220 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:30:35.221 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:30:35.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:30:35.222 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:30:35.222 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:30:35.223 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:30:35.227 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:30:35.228 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T19:32:01.288 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:32:01.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:32:01.304 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T19:32:01.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T19:32:01.299+0000 7fc8e9219700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 83333) UID: 1000 2021-02-20T19:32:01.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:32:10.305 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:32:10.306 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:32:10.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:32:10.307 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:32:10.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:32:10.307 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:32:10.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:32:10.311 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T19:32:12.229 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:32:12.230 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:32:12.230 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T19:32:12.231 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T19:32:12.227+0000 7ff857586700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 85193) UID: 1000 2021-02-20T19:32:12.431 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:32:21.231 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:32:21.238 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:32:21.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:32:21.239 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:32:21.239 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:32:21.240 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:32:21.242 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:32:21.243 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T19:34:05.312 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5302302752299266) > weight (0.25) 2021-02-20T19:34:05.313 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.577664736446572) > weight (0.25) 2021-02-20T19:34:05.313 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7539711987750253) > weight (0.25) 2021-02-20T19:34:05.313 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6403754782812676) > weight (0.25) 2021-02-20T19:34:05.313 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T19:34:20.244 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2943596237257432) > weight (0.25) 2021-02-20T19:34:20.244 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3184836069834933) > weight (0.25) 2021-02-20T19:34:20.244 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7144511581385594) > weight (0.25) 2021-02-20T19:34:20.245 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.803374495829971) > weight (0.25) 2021-02-20T19:34:20.245 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T19:35:49.314 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:35:49.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:35:49.325 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T19:35:49.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T19:35:49.319+0000 7f3fdd14f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 85284) UID: 1000 2021-02-20T19:35:49.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:35:55.326 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:35:55.330 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:35:55.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:35:55.331 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:35:55.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:35:55.331 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:35:55.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:35:55.335 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T19:36:00.246 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:36:00.247 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:36:00.247 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T19:36:00.247 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:36:00.247 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:36:00.248 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T19:36:00.243+0000 7eff7205c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 85375) UID: 1000 2021-02-20T19:36:00.448 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:36:06.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:36:06.349 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:36:06.349 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:36:06.350 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:36:06.352 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:36:06.353 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T19:37:03.336 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.31086230857887776) > weight (0.25) 2021-02-20T19:37:03.366 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:37:03.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:37:03.367 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T19:37:03.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T19:37:03.362+0000 7f7562ccb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 84819) UID: 1000 2021-02-20T19:37:03.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:37:07.368 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:37:07.369 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:37:07.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:37:07.370 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:37:07.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:37:07.370 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:37:07.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:37:07.373 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T19:37:23.353 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6602952203431715) > weight (0.25) 2021-02-20T19:37:23.362 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2539155074424507) > weight (0.25) 2021-02-20T19:37:23.362 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.42557653068198464) > weight (0.25) 2021-02-20T19:37:23.362 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7871977261147873) > weight (0.25) 2021-02-20T19:37:23.362 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T19:38:44.364 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9942007508576662) > weight (0.25) 2021-02-20T19:38:44.371 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7792719006022162) > weight (0.25) 2021-02-20T19:38:44.371 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5021764600174525) > weight (0.25) 2021-02-20T19:38:44.371 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7797895721562499) > weight (0.25) 2021-02-20T19:38:44.372 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T19:38:47.374 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.30487128538214037) > weight (0.25) 2021-02-20T19:38:47.375 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6947319619928453) > weight (0.25) 2021-02-20T19:38:47.375 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9777894496169938) > weight (0.25) 2021-02-20T19:38:47.375 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.39633126769775695) > weight (0.25) 2021-02-20T19:38:47.375 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T19:39:52.373 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7417142648080752) > weight (0.25) 2021-02-20T19:39:52.410 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T19:39:52.411 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T19:39:52.411 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T19:39:52.412 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T19:39:52.407+0000 7f2025131700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 84637) UID: 1000 2021-02-20T19:39:52.612 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:40:01.413 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T19:40:01.413 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:40:01.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:40:01.414 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T19:40:01.414 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T19:40:01.415 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T19:40:01.418 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T19:40:01.418 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T19:40:02.377 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:40:02.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:40:02.378 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T19:40:02.378 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:40:02.378 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:40:02.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T19:40:02.375+0000 7f1014840700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 85469) UID: 1000 2021-02-20T19:40:02.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:40:08.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:40:08.490 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:40:08.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:40:08.492 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:40:08.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:40:08.495 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T19:41:23.420 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2879237909588134) > weight (0.25) 2021-02-20T19:41:23.432 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8734408538438542) > weight (0.25) 2021-02-20T19:41:23.432 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7934685013804911) > weight (0.25) 2021-02-20T19:41:23.432 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.34359800710200505) > weight (0.25) 2021-02-20T19:41:23.433 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T19:41:55.496 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6389590423663861) > weight (0.25) 2021-02-20T19:41:55.498 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8109300927127466) > weight (0.25) 2021-02-20T19:41:55.498 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7141739834212587) > weight (0.25) 2021-02-20T19:41:55.499 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.33196604965227716) > weight (0.25) 2021-02-20T19:41:55.499 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T19:42:46.434 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5213391680480775) > weight (0.25) 2021-02-20T19:42:46.470 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3364866337303042) > weight (0.25) 2021-02-20T19:42:46.470 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7779944397167075) > weight (0.25) 2021-02-20T19:42:46.471 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T19:42:46.472 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T19:42:46.472 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T19:42:46.473 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T19:42:46.464+0000 7f6d6ae53700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 85002) UID: 1000 2021-02-20T19:42:46.673 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:42:49.473 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T19:42:49.474 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:42:49.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:42:49.475 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T19:42:49.475 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T19:42:49.475 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T19:42:49.478 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T19:42:49.479 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T19:43:49.500 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:43:49.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:43:49.502 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T19:43:49.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T19:43:49.499+0000 7ff04df19700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 85836) UID: 1000 2021-02-20T19:43:49.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:43:54.503 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:43:54.503 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:43:54.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:43:54.504 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:43:54.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:43:54.505 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:43:54.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:43:54.512 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T19:43:55.479 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:43:55.481 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:43:55.481 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T19:43:55.482 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T19:43:55.479+0000 7f0790137700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 85560) UID: 1000 2021-02-20T19:43:55.682 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:43:58.482 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:43:58.483 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:43:58.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:43:58.483 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:43:58.484 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:43:58.484 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:43:58.487 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:43:58.487 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T19:45:24.513 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:45:25.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:45:26.000 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T19:45:26.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T19:45:25.997+0000 7f4ce9df9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 86088) UID: 1000 2021-02-20T19:45:26.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:45:30.001 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:45:30.002 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:45:30.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:45:30.002 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:45:30.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:45:30.003 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:45:30.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:45:30.006 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T19:45:37.487 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:45:37.488 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:45:37.488 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T19:45:37.489 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T19:45:37.485+0000 7f6264166700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 86179) UID: 1000 2021-02-20T19:45:37.690 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:45:46.489 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:45:46.490 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:45:46.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:45:46.491 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:45:46.491 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:45:46.491 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:45:46.494 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:45:46.494 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T19:47:02.007 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:47:02.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:47:02.018 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T19:47:02.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T19:47:02.016+0000 7fd83ffa5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 86273) UID: 1000 2021-02-20T19:47:02.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:47:10.019 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:47:10.026 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:47:10.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:47:10.027 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:47:10.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:47:10.029 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:47:10.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:47:10.033 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T19:47:16.496 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7374337047029125) > weight (0.25) 2021-02-20T19:47:16.500 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9437587748120452) > weight (0.25) 2021-02-20T19:47:16.500 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9884939214613844) > weight (0.25) 2021-02-20T19:47:16.500 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T19:47:16.501 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T19:47:16.501 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T19:47:16.502 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:2021-02-20T19:47:16.500+0000 7f1d135d6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 85929) UID: 1000 2021-02-20T19:47:16.702 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:47:23.502 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T19:47:23.511 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:47:23.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:47:23.512 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T19:47:23.513 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T19:47:23.513 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T19:47:23.517 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T19:47:23.518 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T19:47:42.944 INFO:tasks.ceph.cluster2.osd.1.smithi090.stderr:2021-02-20T19:47:42.939+0000 7f38efdb0700 -1 reset not still connected to 0x56498340cf20 2021-02-20T19:48:26.519 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:48:26.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:48:26.532 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T19:48:26.533 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:2021-02-20T19:48:26.530+0000 7fe8e0703700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 86364) UID: 1000 2021-02-20T19:48:26.734 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:48:31.534 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:48:31.534 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:48:31.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:48:31.535 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:48:31.535 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:48:31.536 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:48:31.538 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:48:31.539 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T19:48:52.034 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5469139256812956) > weight (0.25) 2021-02-20T19:48:52.035 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9349331849269356) > weight (0.25) 2021-02-20T19:48:52.035 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6753993759935633) > weight (0.25) 2021-02-20T19:48:52.035 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5679919036580848) > weight (0.25) 2021-02-20T19:48:52.036 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T19:49:47.540 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3794869744454795) > weight (0.25) 2021-02-20T19:49:47.547 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6737446776991617) > weight (0.25) 2021-02-20T19:49:47.548 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T19:49:47.548 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T19:49:47.549 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T19:49:47.550 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:2021-02-20T19:49:47.545+0000 7f2039680700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 82874) UID: 1000 2021-02-20T19:49:47.951 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:49:51.820 INFO:tasks.ceph.cluster1.osd.2.smithi003.stderr:2021-02-20T19:49:51.817+0000 7fb533c45700 -1 reset not still connected to 0x558b95d3e700 2021-02-20T19:49:56.549 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T19:49:56.550 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:49:56.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:49:56.551 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T19:49:56.551 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T19:49:56.551 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T19:49:56.554 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T19:49:56.555 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T19:50:02.036 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:50:02.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:50:02.038 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T19:50:02.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T19:50:02.033+0000 7f09d32d8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 86455) UID: 1000 2021-02-20T19:50:02.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:50:09.038 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:50:09.038 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:50:09.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:50:09.039 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:50:09.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:50:09.040 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:50:09.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:50:09.043 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T19:51:30.044 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:51:30.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:51:30.154 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T19:51:30.154 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:51:30.154 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:51:30.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T19:51:30.151+0000 7f0d35380700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 86821) UID: 1000 2021-02-20T19:51:30.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:51:36.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:51:36.257 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:51:36.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:51:36.258 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:51:36.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:51:36.262 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T19:51:41.556 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4976152644123978) > weight (0.25) 2021-02-20T19:51:41.556 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5923990866089136) > weight (0.25) 2021-02-20T19:51:41.557 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6755552403881646) > weight (0.25) 2021-02-20T19:51:41.557 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8158240167855647) > weight (0.25) 2021-02-20T19:51:41.557 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T19:52:40.263 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5345116148309015) > weight (0.25) 2021-02-20T19:52:40.270 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:52:40.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:52:40.271 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T19:52:40.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T19:52:40.270+0000 7f3c6e25e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 85651) UID: 1000 2021-02-20T19:52:40.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:52:49.273 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:52:49.279 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:52:49.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:52:49.280 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:52:49.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:52:49.280 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:52:49.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:52:49.283 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T19:53:16.558 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5043259118588229) > weight (0.25) 2021-02-20T19:53:16.566 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T19:53:16.567 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T19:53:16.567 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T19:53:16.568 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:2021-02-20T19:53:16.561+0000 7feea082a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 85745) UID: 1000 2021-02-20T19:53:16.969 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:53:20.568 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T19:53:20.568 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:53:20.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:53:20.569 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T19:53:20.569 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T19:53:20.570 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T19:53:20.574 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T19:53:20.574 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T19:54:14.285 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:54:14.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:54:14.292 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T19:54:14.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:2021-02-20T19:54:14.288+0000 7f08e4552700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 86912) UID: 1000 2021-02-20T19:54:14.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:54:15.293 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:54:15.293 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:54:15.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:54:15.294 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:54:15.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:54:15.295 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:54:15.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:54:15.298 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T19:54:25.576 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.31281524226615176) > weight (0.25) 2021-02-20T19:54:25.576 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4342890993350408) > weight (0.25) 2021-02-20T19:54:25.576 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.513646023056255) > weight (0.25) 2021-02-20T19:54:25.577 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6925371702258857) > weight (0.25) 2021-02-20T19:54:25.577 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T19:55:37.299 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3090714148644915) > weight (0.25) 2021-02-20T19:55:37.361 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:55:37.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:55:37.362 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T19:55:37.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:2021-02-20T19:55:37.359+0000 7fef77f0e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 87003) UID: 1000 2021-02-20T19:55:37.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi090.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:55:46.364 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:55:46.364 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:55:46.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:55:46.365 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:55:46.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:55:46.366 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:55:46.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:55:46.369 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T19:55:50.204 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2021-02-20T19:55:50.208 DEBUG:teuthology.task.console_log:Killing console logger for smithi003 2021-02-20T19:55:50.210 DEBUG:teuthology.task.console_log:Killing console logger for smithi090 2021-02-20T19:55:50.211 DEBUG:teuthology.exit:Finished running handlers