Merge pull request #210 from fog/upgrade-google-client

Upgrade google client to 0.15

Author: icco

.rubocop.yml

@@ -6,9 +6,6 @@ Style/ConditionalAssignment:
   SingleLineConditionsOnly: true
   EnforcedStyle: assign_inside_condition
 
-Style/Encoding:
-  EnforcedStyle: when_needed
-
 Style/FormatString:
   Enabled: false
 
@@ -40,15 +37,15 @@ Style/CollectionMethods:
     find: detect
     find_all: select
     reduce: inject
-Style/DotPosition:
+Layout/DotPosition:
   Description: Checks the position of the dot in multi-line method calls.
   StyleGuide: https://github.com/bbatsov/ruby-style-guide#consistent-multi-line-chains
   Enabled: true
   EnforcedStyle: leading
   SupportedStyles:
   - leading
   - trailing
-Style/FileName:
+Naming/FileName:
   Description: Use snake_case for source file names.
   StyleGuide: https://github.com/bbatsov/ruby-style-guide#snake-case-files
   Enabled: false
@@ -80,7 +77,7 @@ Style/PercentLiteralDelimiters:
     "%w": "()"
     "%W": "()"
     "%x": "()"
-Style/PredicateName:
+Naming/PredicateName:
   Description: Check the names of predicate methods.
   StyleGuide: https://github.com/bbatsov/ruby-style-guide#bool-methods-qmark
   Enabled: true
@@ -186,7 +183,7 @@ Lint/AssignmentInCondition:
 Style/InlineComment:
   Description: Avoid inline comments.
   Enabled: true
-Style/AccessorMethodName:
+Naming/AccessorMethodName:
   Description: Check the naming of accessor methods for get_/set_.
   Enabled: false
 Style/Alias:

README.md

@@ -6,7 +6,7 @@ The main maintainers for the Google sections are @icco, @Temikus and @plribeiro3
 
 **As of v0.1.1, Google no longer supports Ruby versions less than 2.0.0.**
 
-**Currently, `fog-google` does not support versions of `google-api-client` >= 0.9 or <= 0.8.5.**
+**As of v1.0.0, fog-google includes google-api-client as a dependency**
 
 
 ## Storage
@@ -20,24 +20,20 @@ There are two ways to access [Google Cloud Storage](https://cloud.google.com/sto
 
 Google Compute Engine is a Virtual Machine hosting service. Currently it is built on version [v1](https://developers.google.com/compute/docs/reference/v1/) of the GCE API.
 
-As of 2015-12-07, we believe Fog for Google Compute engine (`Fog::Compute::Google`) is feature complete.
-
-If you are using Fog to interact with GCE, please keep Fog up to date and [file issues](https://github.com/fog/fog-google/issues) for any anomalies you see or features you would like.
+As of 2017-12-15, we are still working on making Fog for Google Compute engine (`Fog::Compute::Google`) feature complete. If you are using Fog to interact with GCE, please keep Fog up to date and [file issues](https://github.com/fog/fog-google/issues) for any anomalies you see or features you would like.
 
 ## SQL
 
-Fog implements [v1beta3](https://cloud.google.com/sql/docs/admin-api/v1beta3/) of the Google Cloud SQL Admin API. This is a currently deprecated API. Pull Requests for updates would be greatly appreciated.
+Fog implements [v1beta4](https://cloud.google.com/sql/docs/mysql/admin-api/v1beta4/) of the Google Cloud SQL Admin API. As of 2017-11-06, Cloud SQL is mostly feature-complete. Please [file issues](https://github.com/fog/fog-google/issues) for any anomalies you see or features you would like as we finish 
+adding remaining features. 
 
 ## DNS
 
 Fog implements [v1](https://cloud.google.com/dns/api/v1/) of the Google Cloud DNS API. We are always looking for people to improve our code and test coverage, so please [file issues](https://github.com/fog/fog-google/issues) for any anomalies you see or features you would like.
 
 ## Monitoring
 
-Fog mostly implements [v3](https://cloud.google.com/monitoring/api/ref_v3/rest/) of the Google Cloud Monitoring API. 
-
-As of 2017-09-26, some less common API methods are missing and may be added as requested. Feature requests or pull requests for 
-additions are welcome. 
+Fog implements [v3](https://cloud.google.com/monitoring/api/v3/) of the Google Cloud Monitoring API. As of 2017-10-05, we believe Fog for Google Cloud Monitoring is feature complete for metric-related resources and are working on supporting groups. 
 
 We are always looking for people to improve our code and test coverage, so please [file issues](https://github.com/fog/fog-google/issues) for any anomalies you see or features you would like.
 
@@ -46,7 +42,7 @@ We are always looking for people to improve our code and test coverage, so pleas
 Note: You **must** have a version of google-api-client > 0.8.5 to use the Pub/Sub API; previous versions will not work.
 
 Fog mostly implements [v1](https://cloud.google.com/pubsub/reference/rest/) of the Google Cloud Pub/Sub API; however some less common API methods are missing. Pull requests for additions would be greatly appreciated.
- 
+
 ## Installation
 
 Add the following two lines to your application's `Gemfile`:
@@ -68,16 +64,22 @@ Or install it yourself as:
 $ gem install fog-google
 ```
 
+## Testing
+
+The tests in `tests` are deprecated. We are currently working on a migration of tests to `minitest`.
+
+For your test to be tested with real credentials, a repo maintainer may add the label `integrate` to your PR to run integration tests.
+
 ## Setup
 
 #### Credentials
 
 Follow the [instructions to generate a private key](https://cloud.google.com/storage/docs/authentication#generating-a-private-key). A sample credentials file can be found in `.fog.example` in this directory:
 
-    cat .fog.example >> ~/.fog # appends the sample configuration
-    vim ~/.fog                 # edit file with yout config
-    
-
+```
+cat .fog.example >> ~/.fog # appends the sample configuration
+vim ~/.fog                 # edit file with yout config
+```
 
 #### SSH-ing into instances
 
@@ -99,4 +101,4 @@ $ bundle exec pry
 
 ## Contributing
 
-See `CONTRIBUTING.md` in this repository.
+See `CONTRIBUTING.md` in this repository.

ci/.gitignore

@@ -0,0 +1 @@
+credentials.yml

ci/README.md

@@ -0,0 +1,70 @@
+# fog-google CI
+
+This pipeline performs two functions.
+
+1. Performing an integration test of the
+upgrade-google-client branch whenever new changes appear there.
+
+1. Running integration tests against every PR with the
+`integrate` label. This allows whitelisted PRs to be tested
+automatically before being merged. Status is updated on the
+PR when the test completes.
+
+## Setup
+
+In order to run the fog-google Concourse Pipeline you must have an existing
+[Concourse](http://concourse.ci) environment.
+See [Deploying Concourse on Google Compute Engine](https://github.com/cloudfoundry-incubator/bosh-google-cpi-release/blob/master/docs/deploy_concourse.md)
+for instructions.
+
+To deploy the pipeline:
+
+* Download the `fly` binary from the Concourse web interface and place it in your `$PATH`.
+
+* Login to your Concourse:
+
+```
+fly -t fog-google login -c <YOUR CONCOURSE URL>
+```
+
+* Update the [credentials.yml](https://github.com/fog/fog-google/blob/master/ci/credentials.yml.tpl)
+file. See [Credentials Requirements](#credentials-requirements) for specific instructions.
+
+* Set the fog-google pipeline:
+
+```
+fly -t fog-google set-pipeline -p fog-google -c pipeline.yml -l credentials.yml
+```
+
+* Unpause the fog-google pipeline:
+
+```
+fly -t fog-google unpause-pipeline -p fog-google
+```
+
+## Credentials Requirements
+
+Several external pieces of authentication are needed for credentials.yml
+
+1. A JSON Service Account File for a service account with at least Editor access to the project.
+    * To get a Service Account File, see [here](https://developers.google.com/identity/protocols/OAuth2ServiceAccount#creatinganaccount) 
+      and create using the Project/Editor role.
+
+1. A [Github Access Token](https://github.com/blog/1509-personal-api-tokens) with at least
+`repo:status` access and a private key with push access to the `fog/fog-google` repositoy.
+
+These items are equivalent to your login credentials for their resources.
+
+## Login Gotchas
+
+* If your Concourse deployment is using a self-signed certificate, use `--insecure` to
+trust the provided certificate.
+
+* If logging into a specific team, ie `fog-google`, use `--team-name fog-google` to specify that.
+
+## Dockerfile
+
+The [docker-image](https://github.com/fog/fog-google/blob/master/ci/docker-image) directory contains
+the Dockerfile necessary for recreating the Docker image used in tasks. This is referenced
+in a per-task basis as `image_resource`, ie in
+image_resource [run-int.yml](https://github.com/fog/fog-google/blob/master/ci/tasks/run-int.yml)

ci/credentials.yml.tpl

@@ -0,0 +1,13 @@
+---
+# Google Cloud Platform project to run under
+google_project:
+# Google Compute Engine Service Account email
+google_client_email:
+# Google Compute Engine Service Account JSON
+google_json_key_data: |
+
+# An access token with repo:status access
+github_access_token:
+# A SSH private key associated with the access token's account or
+# a repo push deploy key
+github_private_key: |

ci/docker-image/Dockerfile

@@ -0,0 +1,20 @@
+FROM ubuntu:16.04
+
+# Packages
+RUN DEBIAN_FRONTEND=noninteractive apt-get -y -qq update && apt-get -y -qq install \
+  build-essential \
+  curl \
+  git \
+  zlib1g-dev \
+  libssl-dev \
+  libreadline-dev \
+  libyaml-dev \
+  libxml2-dev \
+  libxslt-dev
+
+# Ubuntu 16.04 will fetch us 2.3.x without any issues.
+RUN DEBIAN_FRONTEND=noninteractive apt-get -y -qq install ruby ruby-dev
+
+RUN apt-get clean
+
+RUN gem install bundler

ci/pipeline.yml

@@ -0,0 +1,63 @@
+jobs:
+  - name: base-int
+    plan:
+      - aggregate:
+        - {
+            get: fog-google-src, resource: fog-google-src-in, trigger: true
+          }
+
+      - task: full-integration-tests
+        file: fog-google-src/ci/tasks/run-int.yml
+        params:
+          google_project: {{google_project}}
+          google_json_key_data: {{google_json_key_data}}
+          google_client_email: {{google_client_email}}
+
+  - name: pr-int
+    plan:
+      - get: fog-google-src
+        resource: pull-request
+        version: every
+        trigger: true
+      - put: pull-request
+        params: {path: fog-google-src, status: pending}
+
+      - task: full-integration-tests
+        file: fog-google-src/ci/tasks/run-int.yml
+        params:
+          google_project: {{google_project}}
+          google_json_key_data: {{google_json_key_data}}
+          google_client_email: {{google_client_email}}
+        on_success:
+          put: pull-request
+          params:
+            path: fog-google-src
+            status: success
+        on_failure:
+          put: pull-request
+          params:
+            path: fog-google-src
+            status: failure
+
+resources:
+  - name: fog-google-src-in
+    type: git
+    source:
+      uri: https://github.com/fog/fog-google.git
+      branch: upgrade-google-client
+  - name: pull-request
+    type: pull-request
+    source:
+      access_token: {{github_access_token}}
+      private_key: {{github_private_key}}
+      label: integrate
+      repo: fog/fog-google
+
+resource_types:
+  # This is a third party resource.
+  # The referenced docker-image is maintained externally
+  # https://github.com/jtarchie/pullrequest-resource
+  - name: pull-request
+    type: docker-image
+    source:
+      repository: jtarchie/pr

ci/tasks/run-int.sh

@@ -0,0 +1,34 @@
+#!/usr/bin/env bash
+
+set -e
+
+my_dir="$( cd $(dirname $0) && pwd )"
+release_dir="$( cd ${my_dir} && cd ../.. && pwd )"
+workspace_dir="$( cd ${release_dir} && cd ../../../.. && pwd )"
+
+pushd ${release_dir} > /dev/null
+
+source ci/tasks/utils.sh
+
+popd > /dev/null
+
+check_param google_project
+check_param google_client_email
+check_param google_json_key_data
+
+echo $google_json_key_data > `pwd`/service_account_key.json
+
+cat >~/.fog <<EOL
+test:
+  google_project: ${google_project}
+  google_client_email: ${google_client_email}
+  google_json_key_location: `pwd`/service_account_key.json
+EOL
+
+pushd ${release_dir} > /dev/null
+
+bundle install
+
+FOG_MOCK=false rake test
+
+popd > /dev/null

ci/tasks/run-int.yml

@@ -0,0 +1,15 @@
+---
+platform: linux
+image_resource:
+  type: docker-image
+  source: {repository: everlag/fog-google}
+inputs:
+- name: fog-google-src
+  path: src/fog-google
+run:
+  path: src/fog-google/ci/tasks/run-int.sh
+params:
+  google_project: replace-me
+  google_client_email: replace-me
+  google_json_key_data: |
+    replace-me

ci/tasks/utils.sh

@@ -0,0 +1,10 @@
+#!/usr/bin/env bash
+
+check_param() {
+  local name=$1
+  local value=$(eval echo '$'$name)
+  if [ "$value" == 'replace-me' ]; then
+    echo "environment variable $name must be set"
+    exit 1
+  fi
+}