All Projects → googleapis → Google Cloud Java

googleapis / Google Cloud Java

Licence: apache-2.0
Google Cloud Client Library for Java

Programming Languages

java
68154 projects - #9 most used programming language
CSS
56736 projects
HTML
75241 projects
shell
77523 projects
python
139335 projects - #7 most used programming language
Dockerfile
14818 projects

Projects that are alternatives of or similar to Google Cloud Java

Google Cloud Go
Google Cloud Client Libraries for Go.
Stars: ✭ 2,720 (+69.36%)
Mutual labels:  google-cloud, cloud-storage, cloud-bigquery, cloud-datastore
storage-abstraction
Provides an abstraction layer for interacting with a storage; the storage can be local or in the cloud.
Stars: ✭ 36 (-97.76%)
Mutual labels:  google-cloud, cloud-storage
iris3
An upgraded and improved version of the Iris automatic GCP-labeling project
Stars: ✭ 38 (-97.63%)
Mutual labels:  google-cloud, cloud-storage
universal-translator
Demo app using Firebase and Google Cloud products to implement a "universal translator"
Stars: ✭ 37 (-97.7%)
Mutual labels:  google-cloud, cloud-storage
Continuous Deployment On Kubernetes
Get up and running with Jenkins on Google Kubernetes Engine
Stars: ✭ 1,494 (-6.97%)
Mutual labels:  google-cloud
Bank Vaults
A Vault swiss-army knife: a K8s operator, Go client with automatic token renewal, automatic configuration, multiple unseal options and more. A CLI tool to init, unseal and configure Vault (auth methods, secret engines). Direct secret injection into Pods.
Stars: ✭ 1,316 (-18.06%)
Mutual labels:  google-cloud
Community
This repository holds the content submitted to https://cloud.google.com/community. Files added to the tutorials/ directory will appear at https://cloud.google.com/community/tutorials.
Stars: ✭ 1,306 (-18.68%)
Mutual labels:  google-cloud
Ydcmd
Консольный клиент Linux/FreeBSD для работы с Яндекс.Диск (Yandex.Disk) посредством REST API
Stars: ✭ 87 (-94.58%)
Mutual labels:  cloud-storage
Teammate Android
A Team Management app for creating tournaments and games for various sports
Stars: ✭ 116 (-92.78%)
Mutual labels:  google-cloud
Datasplash
Clojure API for a more dynamic Google Dataflow
Stars: ✭ 111 (-93.09%)
Mutual labels:  google-cloud
Typhoon
Minimal and free Kubernetes distribution with Terraform
Stars: ✭ 1,397 (-13.01%)
Mutual labels:  google-cloud
Magnolify
A collection of Magnolia add-on modules
Stars: ✭ 81 (-94.96%)
Mutual labels:  google-cloud
Keras Cloud Ml Engine
Adventures using keras on Google's Cloud ML Engine
Stars: ✭ 106 (-93.4%)
Mutual labels:  google-cloud
Hoarder
A simple, api-driven storage system for storing code builds and cached libraries for cloud-based deployment services.
Stars: ✭ 91 (-94.33%)
Mutual labels:  cloud-storage
Qwiklabs
labs guide for completing qwiklabs challenge
Stars: ✭ 103 (-93.59%)
Mutual labels:  google-cloud
S3git
s3git: git for Cloud Storage. Distributed Version Control for Data. Create decentralized and versioned repos that scale infinitely to 100s of millions of files. Clone huge PB-scale repos on your local SSD to make changes, commit and push back. Oh yeah, it dedupes too and offers directory versioning.
Stars: ✭ 1,287 (-19.86%)
Mutual labels:  cloud-storage
Jhipster Microservices Example
JHipster Microservices Example using Spring Cloud, Spring Boot, Angular, Docker, and Kubernetes
Stars: ✭ 100 (-93.77%)
Mutual labels:  google-cloud
Microservices Demo
Sample cloud-native application with 10 microservices showcasing Kubernetes, Istio, gRPC and OpenCensus.
Stars: ✭ 11,369 (+607.91%)
Mutual labels:  google-cloud
Iap Desktop
IAP Desktop is a Windows application that provides zero-trust Remote Desktop and SSH access to Linux and Windows VMs on Google Cloud.
Stars: ✭ 96 (-94.02%)
Mutual labels:  google-cloud
Unity Solutions
Use Firebase tools to incorporate common features into your games!
Stars: ✭ 95 (-94.08%)
Mutual labels:  google-cloud

Google Cloud Java Client Libraries

Java idiomatic client for Google Cloud Platform services.

Supported APIs

Libraries are available on GitHub and Maven Central for developing Java applications that interact with individual Google Cloud services:

Client Release Level Version
API Gateway GA Maven
Access Approval GA Maven
Apigee Connect GA Maven
App Engine Admin API GA Maven
Asset Inventory GA Maven
Auto ML GA Maven
BigQuery GA Maven
BigQuery Connection GA Maven
BigQuery Data Transfer Service GA Maven
BigQuery Reservation GA Maven
BigQuery Storage GA Maven
Bigtable GA Maven
Bigtable Hbase Client GA Maven
Billing GA Maven
Billing Budgets GA Maven
Build GA Maven
CCAI Insights GA Maven
Certificate Authority Service GA Maven
Channel Services GA Maven
Composer GA Maven
Container Analysis GA Maven
DNS GA Maven
Data Catalog GA Maven
Data Loss Prevention GA Maven
Database Migration Service GA Maven
Dataproc GA Maven
Dataproc Metastore GA Maven
Datastore GA Maven
Debugger GA Maven
Dialogflow API GA Maven
Document AI GA Maven
Essential Contacts API GA Maven
Eventarc GA Maven
Filestore API GA Maven
Firestore GA Maven
Functions GA Maven
Gaming GA Maven
IAM Admin API GA Maven
IAM Service Account Credentials API GA Maven
Identity Access Context Manager GA Maven
Internet of Things (IoT) Core GA Maven
Key Management Service GA Maven
Kubernetes Engine GA Maven
Logging GA Maven
Memcache GA Maven
Monitoring Dashboards GA Maven
Natural Language GA Maven
Network Management API GA Maven
OS Config API GA Maven
OS Login GA Maven
Organization Policy GA Maven
Profiler GA Maven
Pub/Sub GA Maven
Pub/Sub Lite GA Maven
Recommender GA Maven
Redis GA Maven
Resource Manager API GA Maven
Resource Settings API GA Maven
Retail GA Maven
Scheduler GA Maven
Secret Management GA Maven
Security Command Center GA Maven
Security Scanner GA Maven
Serverless VPC Access GA Maven
Service Control API GA Maven
Service Directory GA Maven
Service Management API GA Maven
Service Usage GA Maven
Shell GA Maven
Spanner GA Maven
Spanner JDBC GA Maven
Speech GA Maven
Stackdriver Monitoring GA Maven
Stackdriver Trace GA Maven
Storage GA Maven
TPU GA Maven
Talent Solution GA Maven
Tasks GA Maven
Text-to-Speech GA Maven
Translation GA Maven
Vertex AI GA Maven
Video Intelligence GA Maven
Vision GA Maven
Web Risk GA Maven
Workflow Executions GA Maven
Workflows GA Maven
Workspace Add-ons API GA Maven
reCAPTCHA Enterprise GA Maven
AI Platform Notebooks Beta Maven
Artifact Registry Beta Maven
Assured Workloads for Government Beta Maven
Binary Authorization Beta Maven
Compute Engine Beta Maven
Connect Gateway API Beta Maven
Data Fusion Beta Maven
Data Labeling Beta Maven
Dataflow Beta Maven
Deploy Beta Maven
Dialogflow CX Beta Maven
Domains Beta Maven
Error Reporting Beta Maven
GKE Hub API Beta Maven
IAM Policy Troubleshooter API Beta Maven
Intrusion Detection System Beta Maven
Life Sciences Beta Maven
Managed Service for Microsoft Active Directory Beta Maven
Media Translation API Beta Maven
NIO Filesystem Provider for Storage Beta Maven
Network Security API Beta Maven
Phishing Protection Beta Maven
Private Catalog Beta Maven
Pub/Sub Lite Kafka Shim Beta Maven
Recommendations AI Beta Maven
Security Command Center Settings API Beta Maven
Storage Transfer Service Beta Maven
VM Migration Beta Maven
Video Transcoder Beta Maven
Analytics Admin Alpha Maven
Analytics Data Alpha Maven
Area 120 Tables Alpha Maven
BigQuery Migration Alpha Maven
Datastream Alpha Maven
Logging Logback Appender Alpha Maven
Logging Servlet Initializer Alpha Maven
Network Connectivity Center Alpha Maven
Pub/Sub Lite Flink Connector Alpha Maven
Pub/Sub Lite Spark Connector Alpha Maven

If the service is not listed, google-api-java-client interfaces with additional Google Cloud APIs using a legacy REST interface.

When building Java applications, preference should be given to the libraries listed in the table.

Specifying a Project ID

Most google-cloud libraries require a project ID. There are multiple ways to specify this project ID.

  1. When using google-cloud libraries from within Compute/App Engine, there's no need to specify a project ID. It is automatically inferred from the production environment.
  2. When using google-cloud elsewhere, you can do one of the following:
  • Supply the project ID when building the service options. For example, to use Datastore from a project with ID "PROJECT_ID", you can write:
Datastore datastore = DatastoreOptions.newBuilder().setProjectId("PROJECT_ID").build().getService();
  • Specify the environment variable GOOGLE_CLOUD_PROJECT to be your desired project ID.
  • Set the project ID using the Google Cloud SDK. To use the SDK, download the SDK if you haven't already, and set the project ID from the command line. For example:
gcloud config set project PROJECT_ID

google-cloud determines the project ID from the following sources in the listed order, stopping once it finds a value:

  1. The project ID supplied when building the service options
  2. Project ID specified by the environment variable GOOGLE_CLOUD_PROJECT
  3. The App Engine / Compute Engine project ID
  4. The project ID specified in the JSON credentials file pointed by the GOOGLE_APPLICATION_CREDENTIALS environment variable
  5. The Google Cloud SDK project ID

In cases where the library may expect a project ID explicitly, we provide a helper that can provide the inferred project ID:

  import com.google.cloud.ServiceOptions;
  ...
  String projectId = ServiceOptions.getDefaultProjectId();

Authentication

google-cloud-java uses https://github.com/googleapis/google-auth-library-java to authenticate requests. google-auth-library-java supports a wide range of authentication types; see the project's README and javadoc for more details.

Google Cloud Platform environment

When using Google Cloud libraries from a Google Cloud Platform environment such as Compute Engine, Kubernetes Engine, or App Engine, no additional authentication steps are necessary.

For example:

Storage storage = StorageOptions.getDefaultInstance().getService();

or:

CloudTasksClient cloudTasksClient = CloudTasksClient.create();

Other environments

Using a service account (recommended)

  1. Generate a JSON service account key.

  2. After downloading that key, you must do one of the following:

    • Define the environment variable GOOGLE_APPLICATION_CREDENTIALS to be the location of the key. For example:
    export GOOGLE_APPLICATION_CREDENTIALS=/path/to/my/key.json
    • Supply the JSON credentials file when building the service options. For example, this Storage object has the necessary permissions to interact with your Google Cloud Storage data:
    Storage storage = StorageOptions.newBuilder()
        .setCredentials(ServiceAccountCredentials.fromStream(new FileInputStream("/path/to/my/key.json")))
        .build()
        .getService();

Local development/testing

If running locally for development/testing, you can use the Google Cloud SDK. Create Application Default Credentials with gcloud auth application-default login, and then google-cloud will automatically detect such credentials.

Existing OAuth2 access token

If you already have an OAuth2 access token, you can use it to authenticate (notice that in this case, the access token will not be automatically refreshed):

Credentials credentials = GoogleCredentials.create(new AccessToken(accessToken, expirationTime));
Storage storage = StorageOptions.newBuilder()
    .setCredentials(credentials)
    .build()
    .getService();

or:

Credentials credentials = GoogleCredentials.create(new AccessToken(accessToken, expirationTime));
CloudTasksSettings cloudTasksSettings = CloudTasksSettings.newBuilder()
    .setCredentialProvider(FixedCredentialsProvider.create(credentials))
    .build();
CloudTasksClient cloudTasksClient = CloudTasksClient.create(cloudTasksSettings);

Application Default Credentials

If no credentials are provided, google-cloud will attempt to detect them from the environment using GoogleCredentials.getApplicationDefault() which will search for Application Default Credentials in the following locations (in order):

  1. The credentials file pointed to by the GOOGLE_APPLICATION_CREDENTIALS environment variable
  2. Credentials provided by the Google Cloud SDK gcloud auth application-default login command
  3. Google App Engine built-in credentials
  4. Google Cloud Shell built-in credentials
  5. Google Compute Engine built-in credentials

Troubleshooting

To get help, follow the instructions in the Troubleshooting document.

Configuring a Proxy

HTTP Clients

For HTTP clients, a proxy can be configured by using http.proxyHost, https.proxyHost, and related system properties as documented by Java Networking and Proxies.

gRPC Clients

For gRPC clients, a proxy can be configured by using the GRPC_PROXY_EXP environment variable as documented by the gRPC release notes. Please note that gRPC proxy support is currently experimental.

For a more custom proxy, you will need supply a ProxyDetector to the ManagedChannelBuilder:

import com.google.api.core.ApiFunction;
import com.google.api.gax.rpc.TransportChannelProvider;
import com.google.cloud.tasks.v2.CloudTasksClient;
import com.google.cloud.tasks.v2.CloudTasksSettings;
import com.google.cloud.tasks.v2.stub.CloudTasksStubSettings;
import io.grpc.HttpConnectProxiedSocketAddress;
import io.grpc.ManagedChannelBuilder;
import io.grpc.ProxiedSocketAddress;
import io.grpc.ProxyDetector;

import javax.annotation.Nullable;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.SocketAddress;

public CloudTasksClient getService() throws IOException {
  TransportChannelProvider transportChannelProvider =
      CloudTasksStubSettings.defaultGrpcTransportProviderBuilder()
          .setChannelConfigurator(
              new ApiFunction<ManagedChannelBuilder, ManagedChannelBuilder>() {
                @Override
                public ManagedChannelBuilder apply(ManagedChannelBuilder managedChannelBuilder) {
                  return managedChannelBuilder.proxyDetector(
                      new ProxyDetector() {
                        @Nullable
                        @Override
                        public ProxiedSocketAddress proxyFor(SocketAddress socketAddress)
                            throws IOException {
                          return HttpConnectProxiedSocketAddress.newBuilder()
                              .setUsername(PROXY_USERNAME)
                              .setPassword(PROXY_PASSWORD)
                              .setProxyAddress(new InetSocketAddress(PROXY_HOST, PROXY_PORT))
                              .setTargetAddress((InetSocketAddress) socketAddress)
                              .build();
                        }
                      });
                }
              })
          .build();
  CloudTasksSettings cloudTasksSettings =
      CloudTasksSettings.newBuilder()
          .setTransportChannelProvider(transportChannelProvider)
          .build();
  return CloudTasksClient.create(cloudTasksSettings);
}

Long Running Operations

Long running operations (LROs) are often used for API calls that are expected to take a long time to complete (e.g. provisioning a GCE instance or a Dataflow pipeline). The initial API call creates an "operation" on the server and returns an operation ID to track its progress.

Our generated gRPC clients provide a nice interface for starting the operation and then waiting for the operation to complete. This is accomplished by returning an OperationFuture. When you call get() on the OperationFuture we poll the operation endpoint to check on the operation. These polling operations have a default timeout that varies from service to service and will throw a java.util.concurrent.CancellationException with the message: Task was cancelled. after that timeout has been reached.

Configuring LRO Timeouts

ClusterControllerSettings.Builder settingsBuilder = ClusterControllerSettings.newBuilder();
TimedRetryAlgorithm timedRetryAlgorithm = OperationTimedPollAlgorithm.create(
		RetrySettings.newBuilder()
				.setInitialRetryDelay(Duration.ofMillis(500L))
				.setRetryDelayMultiplier(1.5)
				.setMaxRetryDelay(Duration.ofMillis(5000L))
				.setInitialRpcTimeout(Duration.ZERO) // ignored
				.setRpcTimeoutMultiplier(1.0) // ignored
				.setMaxRpcTimeout(Duration.ZERO) // ignored
				.setTotalTimeout(Duration.ofHours(24L))	// set polling timeout to 24 hours
				.build());
settingsBuilder.createClusterOperationSettings()
		.setPollingAlgorithm(timedRetryAlgorithm);
ClusterControllerClient clusterControllerClient = ClusterControllerClient.create(settingsBuilder.build());

Managing Dependencies

If you are using more than one Google Cloud client library, we recommend you use one of our Bill of Material (BOM) artifacts to help manage dependency versions. For more information, see https://github.com/GoogleCloudPlatform/cloud-opensource-java/wiki/The-Google-Cloud-Platform-Libraries-BOM.

Java Versions

Java 7 or above is required for using the clients in this repository.

Supported Platforms

Clients in this repository use either HTTP or gRPC for the transport layer. All HTTP-based clients should work in all environments.

For clients that use gRPC, the supported platforms are constrained by the platforms that Forked Tomcat Native supports, which for architectures means only x86_64, and for operating systems means Mac OS X, Windows, and Linux. Additionally, gRPC constrains the use of platforms with threading restrictions.

Thus, the following are not supported:

  • Android
    • Consider Firebase, which includes many of these APIs.
    • It is possible to use these libraries in many cases, although it is unsupported. You can find examples, such as this one, in this example repository but consider the risks carefully before using these libraries in an application.
  • Raspberry Pi (since it runs on the ARM architecture)
  • Google App Engine Standard Java 7

The following environments should work (among others):

  • standalone Windows on x86_64
  • standalone Mac OS X on x86_64
  • standalone Linux on x86_64
  • Google Compute Engine (GCE)
  • Google Container Engine (GKE)
  • Google App Engine Standard Java 8 (GAE Std J8)
  • Google App Engine Flex (GAE Flex)
  • Alpine Linux (Java 11+)

Testing

This library provides tools to help write tests for code that uses google-cloud services.

See TESTING to read more about using our testing helpers.

Versioning

This library follows Semantic Versioning, with some additional qualifications:

  1. Components marked with @BetaApi or @Experimental are considered to be "0.x" features inside a "1.x" library. This means they can change between minor and patch releases in incompatible ways. These features should not be used by any library "B" that itself has consumers, unless the components of library B that use @BetaApi features are also marked with @BetaApi. Features marked as @BetaApi are on a path to eventually become "1.x" features with the marker removed.

    Special exception for google-cloud-java: google-cloud-java is allowed to depend on @BetaApi features in gax-java without declaring the consuming code @BetaApi, because gax-java and google-cloud-java move in step with each other. For this reason, gax-java should not be used independently of google-cloud-java.

  2. Components marked with @InternalApi are technically public, but only because of the limitations of Java's access modifiers. For the purposes of semver, they should be considered private.

  3. Interfaces marked with @InternalExtensionOnly are public, but should only be implemented by internal classes. For the purposes of semver, we reserve the right to add to these interfaces without default implementations (for Java 7).

Please note these clients are currently under active development. Any release versioned 0.x.y is subject to backwards incompatible changes at any time.

GA

Libraries defined at a GA quality level are expected to be stable and all updates in the libraries are guaranteed to be backwards-compatible. Any backwards-incompatible changes will lead to the major version increment (1.x.y -> 2.0.0).

Beta

Libraries defined at a Beta quality level are expected to be mostly stable and we're working towards their release candidate. We will address issues and requests with a higher priority.

Alpha

Libraries defined at an Alpha quality level are still a work-in-progress and are more likely to get backwards-incompatible updates. Additionally, it's possible for Alpha libraries to get deprecated and deleted before ever being promoted to Beta or GA.

IDE Plugins

If you're using IntelliJ or Eclipse, you can add client libraries to your project using these IDE plugins:

Besides adding client libraries, the plugins provide additional functionality, such as service account key management. Refer to the documentation for each plugin for more details.

These client libraries can be used on App Engine standard for Java 8 runtime and App Engine flexible (including the Compat runtime). Most of the libraries do not work on the App Engine standard for Java 7 runtime. However, Datastore, Storage, and Bigquery should work.

Contributing

Contributions to this library are always welcome and highly encouraged.

See google-cloud's CONTRIBUTING documentation and the shared documentation for more information on how to get started.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See Code of Conduct for more information.

License

Apache 2.0 - See LICENSE for more information.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].