在上一篇博客我介绍了如何用Testcontainers来做Java测试,并以Kafka为例。在我的项目中,还用到了GCP的一些服务,例如Bigquery,Testcontainers同样提供了对GCP服务的支持。因为官网对于Bigquery的模拟测试介绍的太简略了,我这里总结一下如何来模拟Bigquery。
官网介绍可以用以下代码来配置一个Bigquery container
BigQueryEmulatorContainer container = new BigQueryEmulatorContainer("ghcr.io/goccy/bigquery-emulator:0.4.3")
但是这样得到的container是无法新增一个dataset的,起码我没找到对应的方法。
我这里采用另一种方式来加载Bigquery
public GenericContainer<?> bqSimulator =
new GenericContainer<>(
DockerImageName.parse("ghcr.io/goccy/bigquery-emulator"))
.withExposedPorts(9050,9060)
.withCommand("--project=test-project", "--dataset=dataset1");
这种方式加载,就可指定创建一个名为test-project的项目,并创建一个dataset1
之后就可以按照正常的方式来调用Bigquery的功能了,例如创建table,写入数据,查询数据等。先获取Bigquery client
String url = String.format("http://%s:%d", bqSimulator.getHost(), bqSimulator.getMappedPort(9050));
BigQueryOptions options = BigQueryOptions
.newBuilder()
.setProjectId("test-project")
.setHost(url)
.setLocation(url)
.setCredentials(NoCredentials.getInstance())
.build();
BigQuery bigQuery = options.getService();
查看当前项目的dataset
Page<Dataset> datasets = bigQuery.listDatasets("test-project", DatasetListOption.pageSize(100));
if (datasets == null) {
System.out.println("Dataset does not contain any models");
return;
}
datasets
.iterateAll()
.forEach(
dataset -> System.out.printf("Success! Dataset ID: %s ", dataset.getDatasetId()));
建立一个table,写入测试数据,然后再查询
TableId tableId = TableId.of("dataset1", "abc");
Schema schema =
Schema.of(
Field.of("name", StandardSQLTypeName.STRING),
Field.of("time", StandardSQLTypeName.TIMESTAMP));
TableDefinition tableDefinition = StandardTableDefinition.of(schema);
TableInfo tableInfo = TableInfo.newBuilder(tableId, tableDefinition).build();
bigQuery.create(tableInfo);
DatasetId datasetId = DatasetId.of("test-project", "dataset1");
Page<Table> tables = bigQuery.listTables(datasetId, TableListOption.pageSize(100));
tables.iterateAll().forEach(table -> System.out.print(table.getTableId().getTable() + "\n"));
QueryJobConfiguration jobconf = QueryJobConfiguration.of("insert into dataset1.abc (name, time) values ('test', '2023-01-01T00:00:00')");
bigQuery.query(jobconf);
String queryResult = "Select * from dataset1.abc;";
QueryJobConfiguration queryConfig =
QueryJobConfiguration.newBuilder(queryResult).setUseLegacySql(true).build();
TableResult result = bigQuery.query(queryConfig);
result.iterateAll().forEach(rows -> rows.forEach(row -> System.out.println(row.getValue())));
以上就是对Bigquery进行模拟测试的方法。