Skip to content
This repository has been archived by the owner on Mar 27, 2022. It is now read-only.

bdutil fails to build hdp #65

Open
lvnilesh opened this issue Sep 23, 2015 · 2 comments
Open

bdutil fails to build hdp #65

lvnilesh opened this issue Sep 23, 2015 · 2 comments

Comments

@lvnilesh
Copy link 

gsutil md gs://cloudgenius

bdutil -p beacloudgenius -b cloudgenius -P cg -z us-central1-f -e /home/user/bdutil/platforms/hdp/ambari_env.sh generate_config my.sh

cat my.sh

import_env /home/user/bdutil/platforms/hdp/ambari_env.sh
PROJECT=beacloudgenius
CONFIGBUCKET=cloudgenius
PREFIX=cg
GCE_ZONE=us-central1-f
GCE_MASTER_MACHINE_TYPE=n1-standard-4

bdutil -e my.sh deploy
Tue Sep 22 18:32:19 PDT 2015: Using local tmp dir for staging files: /tmp/bdutil-20150922-183219-iBw
Tue Sep 22 18:32:19 PDT 2015: Using custom environment-variable file(s): bdutil_env.sh my.sh
Tue Sep 22 18:32:19 PDT 2015: Reading environment-variable file: /home/user/bdutil/bdutil_env.sh
Tue Sep 22 18:32:19 PDT 2015: Reading environment-variable file: ./my.sh
Importing dependent env file: /home/user/bdutil/platforms/hdp/ambari_env.sh
Importing dependent env file: /home/user/bdutil/platforms/hdp/ambari_manual_env.sh
Importing dependent env file: /home/user/bdutil/hadoop2_env.sh
Importing dependent env file: /home/user/bdutil/platforms/hdp/ambari.conf
Importing dependent env file: /home/user/bdutil/platforms/hdp/ambari_functions.sh
Deploy cluster with following settings?
      CONFIGBUCKET='cloudgenius'
      PROJECT='beacloudgenius'
      GCE_IMAGE='centos-6'
      GCE_ZONE='us-central1-f'
      GCE_NETWORK='default'
      PREEMPTIBLE_FRACTION=0.0
      PREFIX='cg'
      NUM_WORKERS=4
      MASTER_HOSTNAME='cg-m'
      WORKERS='cg-w-0 cg-w-1 cg-w-2 cg-w-3'
      BDUTIL_GCS_STAGING_DIR='gs://cloudgenius/bdutil-staging/cg-m'
            MASTER_ATTACHED_PD='cg-m-pd'
      WORKER_ATTACHED_PDS='cg-w-0-pd cg-w-1-pd cg-w-2-pd cg-w-3-pd'
      (y/n) y
Tue Sep 22 18:32:21 PDT 2015: Checking for existence of gs://cloudgenius...
gs://cloudgenius/
Tue Sep 22 18:32:21 PDT 2015: Checking for existence of gs://hadoop-dist/hadoop-2.7.1.tar.gz...
Tue Sep 22 18:32:22 PDT 2015: Checking upload files...
Tue Sep 22 18:32:22 PDT 2015: Verified '/home/user/bdutil/conf/hadoop2/core-template.xml'
Tue Sep 22 18:32:22 PDT 2015: Verified '/home/user/bdutil/conf/hadoop2/mapred-template.xml'
Tue Sep 22 18:32:22 PDT 2015: Verified '/home/user/bdutil/conf/hadoop2/yarn-template.xml'
Tue Sep 22 18:32:22 PDT 2015: Verified '/home/user/bdutil/conf/hadoop2/gcs-core-template.xml'
Tue Sep 22 18:32:22 PDT 2015: Verified '/home/user/bdutil/conf/hadoop2/bigtable-hbase-site-template.xml'
Tue Sep 22 18:32:22 PDT 2015: Verified '/home/user/bdutil/conf/hadoop2/hdfs-template.xml'
Tue Sep 22 18:32:22 PDT 2015: Verified '/home/user/bdutil/conf/hadoop2/bq-mapred-template.xml'
Tue Sep 22 18:32:22 PDT 2015: Verified '/home/user/bdutil/libexec/hadoop_helpers.sh'
Tue Sep 22 18:32:22 PDT 2015: Verified '/home/user/bdutil/libexec/configure_mrv2_mem.py'
Tue Sep 22 18:32:22 PDT 2015: Verified '/home/user/bdutil/hadoop2_env.sh'
Tue Sep 22 18:32:22 PDT 2015: Verified '/home/user/bdutil/platforms/hdp/ambari.conf'
Tue Sep 22 18:32:22 PDT 2015: Verified '/home/user/bdutil/platforms/hdp/ambari_functions.sh'
Tue Sep 22 18:32:22 PDT 2015: Verified '/home/user/bdutil/libexec/hadoop_helpers.sh'
Tue Sep 22 18:32:22 PDT 2015: Verified '/home/user/bdutil/platforms/hdp/configuration.json'
Tue Sep 22 18:32:22 PDT 2015: Verified '/home/user/bdutil/platforms/hdp/resources/public-hostname-gcloud.sh'
Tue Sep 22 18:32:22 PDT 2015: Verified '/home/user/bdutil/platforms/hdp/resources/thp-disable.sh'
Tue Sep 22 18:32:22 PDT 2015: Verified '/home/user/bdutil/platforms/hdp/ambari_manual_env.sh'
Tue Sep 22 18:32:22 PDT 2015: Verified '/home/user/bdutil/platforms/hdp/create_blueprint.py'
Tue Sep 22 18:32:22 PDT 2015: Verified '/home/user/bdutil/platforms/hdp/ambari_env.sh'
Tue Sep 22 18:32:22 PDT 2015: Generating 12 command groups...
Tue Sep 22 18:32:22 PDT 2015: Done generating remote shell scripts.
Tue Sep 22 18:32:22 PDT 2015: Creating attached worker disks: cg-w-0-pd cg-w-1-pd cg-w-2-pd cg-w-3-pd
....Tue Sep 22 18:32:23 PDT 2015: Creating attached master disk: cg-m-pd
.Tue Sep 22 18:32:23 PDT 2015: Done creating disks!
Tue Sep 22 18:32:23 PDT 2015: Waiting on async 'disks create' jobs to finish. Might take a while...
.....
Tue Sep 22 18:32:32 PDT 2015: Creating worker instances: cg-w-0 cg-w-1 cg-w-2 cg-w-3
....Tue Sep 22 18:32:32 PDT 2015: Creating master instance: cg-m
.Tue Sep 22 18:32:33 PDT 2015: Waiting on async 'instances create' jobs to finish. Might take a while...
.....
Tue Sep 22 18:33:30 PDT 2015: Instances all created. Entering polling loop to wait for ssh-ability
.....Tue Sep 22 18:33:31 PDT 2015: Waiting on async 'wait_for_ssh' jobs to finish. Might take a while...
.....
Tue Sep 22 18:33:34 PDT 2015: Instances all ssh-able
Tue Sep 22 18:33:34 PDT 2015: Staging file and script dependencies into gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/...
Copying file:///home/user/bdutil/conf/hadoop2/mapred-template.xml [Content-Type=application/xml]...
Copying file:///home/user/bdutil/conf/hadoop2/core-template.xml [Content-Type=application/xml]...
Copying file:///home/user/bdutil/conf/hadoop2/yarn-template.xml [Content-Type=application/xml]...
Copying file:///home/user/bdutil/conf/hadoop2/gcs-core-template.xml [Content-Type=application/xml]...
Copying file:///home/user/bdutil/conf/hadoop2/bigtable-hbase-site-template.xml [Content-Type=application/xml]...
Copying file:///home/user/bdutil/conf/hadoop2/hdfs-template.xml [Content-Type=application/xml]...
Copying file:///home/user/bdutil/conf/hadoop2/bq-mapred-template.xml [Content-Type=application/xml]...
Uploading   ...aging/cg-m/20150922-183219-iBw/core-template.xml: 749 B/749 B    
Copying file:///home/user/bdutil/libexec/hadoop_helpers.sh [Content-Type=text/x-sh]...
Uploading   ...ing/cg-m/20150922-183219-iBw/mapred-template.xml: 5.43 KiB/5.43 KiB    
Uploading   ...aging/cg-m/20150922-183219-iBw/yarn-template.xml: 2.81 KiB/2.81 KiB    
Copying file:///home/user/bdutil/libexec/configure_mrv2_mem.py [Content-Type=text/x-python]...
Copying file:///home/user/bdutil/hadoop2_env.sh [Content-Type=text/x-sh]...
Uploading   ...g/cg-m/20150922-183219-iBw/gcs-core-template.xml: 2.62 KiB/2.62 KiB    
Uploading   ...0922-183219-iBw/bigtable-hbase-site-template.xml: 1.35 KiB/1.35 KiB    
Uploading   ...aging/cg-m/20150922-183219-iBw/hdfs-template.xml: 2.72 KiB/2.72 KiB    
Uploading   .../cg-m/20150922-183219-iBw/bq-mapred-template.xml: 769 B/769 B     
Uploading   ...aging/cg-m/20150922-183219-iBw/hadoop_helpers.sh: 5.98 KiB/5.98 KiB    
Uploading   ...-staging/cg-m/20150922-183219-iBw/hadoop2_env.sh: 2.92 KiB/2.92 KiB    
Uploading   ...g/cg-m/20150922-183219-iBw/configure_mrv2_mem.py: 7.28 KiB/7.28 KiB    
Copying file:///home/user/bdutil/platforms/hdp/ambari.conf [Content-Type=application/octet-stream]...
Copying file:///home/user/bdutil/platforms/hdp/ambari_functions.sh [Content-Type=text/x-sh]...
Copying file:///home/user/bdutil/libexec/hadoop_helpers.sh [Content-Type=text/x-sh]...
Copying file:///home/user/bdutil/platforms/hdp/configuration.json [Content-Type=application/json]...
Copying file:///home/user/bdutil/platforms/hdp/resources/public-hostname-gcloud.sh [Content-Type=text/x-sh]...
Uploading   ...til-staging/cg-m/20150922-183219-iBw/ambari.conf: 2.56 KiB/2.56 KiB    
Copying file:///home/user/bdutil/platforms/hdp/resources/thp-disable.sh [Content-Type=text/x-sh]...
Copying file:///home/user/bdutil/platforms/hdp/ambari_manual_env.sh [Content-Type=text/x-sh]...
Copying file:///home/user/bdutil/platforms/hdp/create_blueprint.py [Content-Type=text/x-python]...
Copying file:///home/user/bdutil/platforms/hdp/ambari_env.sh [Content-Type=text/x-sh]...
Copying file:///tmp/bdutil-20150922-183219-iBw/hadoop-env-setup.sh [Content-Type=text/x-sh]...
Uploading   ...ing/cg-m/20150922-183219-iBw/ambari_functions.sh: 3.62 KiB/3.62 KiB    
Uploading   ...aging/cg-m/20150922-183219-iBw/hadoop_helpers.sh: 5.98 KiB/5.98 KiB    
Uploading   ...-m/20150922-183219-iBw/public-hostname-gcloud.sh: 174 B/174 B    
Uploading   ...ging/cg-m/20150922-183219-iBw/configuration.json: 863 B/863 B    
Uploading   ...ing/cg-m/20150922-183219-iBw/hadoop-env-setup.sh: 30.36 KiB/30.36 KiB    
Uploading   ...-staging/cg-m/20150922-183219-iBw/thp-disable.sh: 451 B/451 B    
Uploading   ...l-staging/cg-m/20150922-183219-iBw/ambari_env.sh: 1.82 KiB/1.82 KiB    
Uploading   ...ing/cg-m/20150922-183219-iBw/create_blueprint.py: 4.12 KiB/4.12 KiB    
Uploading   ...ng/cg-m/20150922-183219-iBw/ambari_manual_env.sh: 3.47 KiB/3.47 KiB    
Copying file:///tmp/bdutil-20150922-183219-iBw/deploy-ssh-master-setup.sh [Content-Type=text/x-sh]...
Copying file:///tmp/bdutil-20150922-183219-iBw/deploy-core-setup.sh [Content-Type=text/x-sh]...
Copying file:///tmp/bdutil-20150922-183219-iBw/deploy-master-nfs-setup.sh [Content-Type=text/x-sh]...
Copying file:///tmp/bdutil-20150922-183219-iBw/deploy-client-nfs-setup.sh [Content-Type=text/x-sh]...
Copying file:///tmp/bdutil-20150922-183219-iBw/deploy-ssh-worker-setup.sh [Content-Type=text/x-sh]...
Copying file:///tmp/bdutil-20150922-183219-iBw/deploy-start.sh [Content-Type=text/x-sh]...
Copying file:///tmp/bdutil-20150922-183219-iBw/install_connectors.sh [Content-Type=text/x-sh]...
Copying file:///tmp/bdutil-20150922-183219-iBw/deploy_start2.sh [Content-Type=text/x-sh]...
Copying file:///tmp/bdutil-20150922-183219-iBw/ambari-setup.sh [Content-Type=text/x-sh]...
Uploading   ...m/20150922-183219-iBw/deploy-ssh-master-setup.sh: 1.59 KiB/1.59 KiB    
Uploading   ...m/20150922-183219-iBw/deploy-client-nfs-setup.sh: 1.53 KiB/1.53 KiB    
Uploading   ...staging/cg-m/20150922-183219-iBw/deploy-start.sh: 1.27 KiB/1.27 KiB    
Uploading   ...m/20150922-183219-iBw/deploy-master-nfs-setup.sh: 3.84 KiB/3.84 KiB    
Uploading   ...m/20150922-183219-iBw/deploy-ssh-worker-setup.sh: 1.37 KiB/1.37 KiB    
Uploading   ...staging/cg-m/20150922-183219-iBw/ambari-setup.sh: 9.22 KiB/9.22 KiB    
Uploading   ...taging/cg-m/20150922-183219-iBw/deploy_start2.sh: 1.41 KiB/1.41 KiB    
Uploading   ...g/cg-m/20150922-183219-iBw/install_connectors.sh: 6.19 KiB/6.19 KiB    
Uploading   ...ng/cg-m/20150922-183219-iBw/deploy-core-setup.sh: 28.41 KiB/28.41 KiB    
Copying file:///tmp/bdutil-20150922-183219-iBw/install-gcs-connector-on-ambari.sh [Content-Type=text/x-sh]...
Uploading   ...22-183219-iBw/install-gcs-connector-on-ambari.sh: 1.47 KiB/1.47 KiB    
Copying file:///tmp/bdutil-20150922-183219-iBw/install-ambari-components.sh [Content-Type=text/x-sh]...
Copying file:///tmp/bdutil-20150922-183219-iBw/update-ambari-config.sh [Content-Type=text/x-sh]...
Uploading   ...20150922-183219-iBw/install-ambari-components.sh: 3.9 KiB/3.9 KiB    
Uploading   ...cg-m/20150922-183219-iBw/update-ambari-config.sh: 2.88 KiB/2.88 KiB    
Tue Sep 22 18:33:36 PDT 2015: Downloading staging files onto VMs...
Tue Sep 22 18:33:36 PDT 2015: Invoking on workers: gcloud --quiet components update gsutil; gsutil -m cp gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/core-template.xml gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/mapred-template.xml gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/yarn-template.xml gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/gcs-core-template.xml gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/bigtable-hbase-site-template.xml gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/hdfs-template.xml gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/bq-mapred-template.xml gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/hadoop_helpers.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/configure_mrv2_mem.py gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/hadoop2_env.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/ambari.conf gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/ambari_functions.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/hadoop_helpers.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/configuration.json gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/public-hostname-gcloud.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/thp-disable.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/ambari_manual_env.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/create_blueprint.py gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/ambari_env.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/hadoop-env-setup.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/deploy-ssh-master-setup.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/deploy-core-setup.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/deploy-master-nfs-setup.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/deploy-client-nfs-setup.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/deploy-ssh-worker-setup.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/deploy-start.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/install_connectors.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/deploy_start2.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/ambari-setup.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/install-gcs-connector-on-ambari.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/update-ambari-config.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/install-ambari-components.sh . && chmod 755 *
....Tue Sep 22 18:33:37 PDT 2015: Invoking on master: gcloud --quiet components update gsutil; gsutil -m cp gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/core-template.xml gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/mapred-template.xml gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/yarn-template.xml gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/gcs-core-template.xml gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/bigtable-hbase-site-template.xml gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/hdfs-template.xml gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/bq-mapred-template.xml gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/hadoop_helpers.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/configure_mrv2_mem.py gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/hadoop2_env.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/ambari.conf gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/ambari_functions.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/hadoop_helpers.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/configuration.json gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/public-hostname-gcloud.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/thp-disable.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/ambari_manual_env.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/create_blueprint.py gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/ambari_env.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/hadoop-env-setup.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/deploy-ssh-master-setup.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/deploy-core-setup.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/deploy-master-nfs-setup.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/deploy-client-nfs-setup.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/deploy-ssh-worker-setup.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/deploy-start.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/install_connectors.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/deploy_start2.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/ambari-setup.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/install-gcs-connector-on-ambari.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/update-ambari-config.sh gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/install-ambari-components.sh . && chmod 755 *
.Tue Sep 22 18:33:37 PDT 2015: Waiting on async 'ssh' jobs to finish. Might take a while...
.....
Tue Sep 22 18:33:51 PDT 2015: Uploads of shell scripts finished, deleting staging files...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/core-template.xml...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/mapred-template.xml...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/yarn-template.xml...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/gcs-core-template.xml...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/bigtable-hbase-site-template.xml...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/hdfs-template.xml...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/bq-mapred-template.xml...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/hadoop_helpers.sh...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/configure_mrv2_mem.py...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/hadoop2_env.sh...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/ambari.conf...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/ambari_functions.sh...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/hadoop_helpers.sh...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/configuration.json...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/public-hostname-gcloud.sh...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/thp-disable.sh...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/ambari_manual_env.sh...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/create_blueprint.py...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/ambari_env.sh...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/hadoop-env-setup.sh...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/deploy-ssh-master-setup.sh...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/deploy-core-setup.sh...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/deploy-master-nfs-setup.sh...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/deploy-client-nfs-setup.sh...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/deploy-ssh-worker-setup.sh...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/deploy-start.sh...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/install_connectors.sh...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/deploy_start2.sh...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/ambari-setup.sh...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/install-gcs-connector-on-ambari.sh...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/update-ambari-config.sh...
Removing gs://cloudgenius/bdutil-staging/cg-m/20150922-183219-iBw/install-ambari-components.sh...
CommandException: 1 files/objects could not be removed.
Tue Sep 22 18:33:55 PDT 2015: Warning. Failed to delete all staging files.
Tue Sep 22 18:33:55 PDT 2015: Invoking on workers: ./ambari-setup.sh
....Tue Sep 22 18:33:56 PDT 2015: Invoking on master: ./ambari-setup.sh
.Tue Sep 22 18:33:56 PDT 2015: Waiting on async 'ssh' jobs to finish. Might take a while...
.....
Tue Sep 22 18:35:34 PDT 2015: Step 'ambari-setup,ambari-setup' done...
Tue Sep 22 18:35:34 PDT 2015: Invoking on master: ./deploy-master-nfs-setup.sh
.Tue Sep 22 18:35:34 PDT 2015: Waiting on async 'ssh' jobs to finish. Might take a while...
.
Tue Sep 22 18:35:41 PDT 2015: Step 'deploy-master-nfs-setup,*' done...
Tue Sep 22 18:35:41 PDT 2015: Invoking on workers: ./deploy-client-nfs-setup.sh
....Tue Sep 22 18:35:42 PDT 2015: Invoking on master: ./deploy-client-nfs-setup.sh
.Tue Sep 22 18:35:42 PDT 2015: Waiting on async 'ssh' jobs to finish. Might take a while...
.....
Tue Sep 22 18:35:55 PDT 2015: Step 'deploy-client-nfs-setup,deploy-client-nfs-setup' done...
Tue Sep 22 18:35:55 PDT 2015: Invoking on master: ./install-ambari-components.sh
.Tue Sep 22 18:35:55 PDT 2015: Waiting on async 'ssh' jobs to finish. Might take a while...
Tue Sep 22 19:36:14 PDT 2015: Exited 1 : gcloud --project=beacloudgenius --quiet --verbosity=info compute ssh cg-m --command=sudo su -l -c "cd ${PWD} && ./install-ambari-components.sh" 2>>install-ambari-components_deploy.stderr 1>>install-ambari-components_deploy.stdout --ssh-flag=-tt --ssh-flag=-oServerAliveInterval=60 --ssh-flag=-oServerAliveCountMax=3 --ssh-flag=-oConnectTimeout=30 --zone=us-central1-f
Tue Sep 22 19:36:14 PDT 2015: Fetching on-VM logs from cg-m
Warning: Permanently added '104.197.16.133' (RSA) to the list of known hosts.
Tue Sep 22 19:36:16 PDT 2015: Command failed: wait ${SUBPROC} on line 326.
Tue Sep 22 19:36:16 PDT 2015: Exit code of failed command: 1
Tue Sep 22 19:36:16 PDT 2015: Detailed debug info available in file: /tmp/bdutil-20150922-183219-iBw/debuginfo.txt
Tue Sep 22 19:36:16 PDT 2015: Check console output for error messages and/or retry your command.
(foobar)
@lvnilesh
Copy link
Author

/tmp/bdutil-20150922-183219-iBw/debuginfo.txt is in this gist

https://gist.github.com/lvnilesh/bafce5637e31efb8dd68

@dennishuo
Copy link
Contributor

Sorry for the delay, must have missed the notification. Did you hit this error consistently, or was it just once? If you try again and hit the error, can you try running the SSH-tunnel commands on https://github.com/GoogleCloudPlatform/bdutil/tree/master/platforms/hdp and then visiting master-hostname:8080 through a browser with username/password 'admin/admin' and seeing if Ambari's UI reports any errors?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lvnilesh @dennishuo