I want to know if it is possible to execute MergeOperation snippet:
MergeOperation mergeOperation = Aggregation.merge()
.intoCollection("someCollection")
.on("_id")
.whenMatched(MergeOperation.WhenDocumentsMatch.mergeDocuments())
.whenNotMatched(MergeOperation.WhenDocumentsDontMatch.discardDocument())
.build();
with the ReactiveMongoTemplate. Like what specified in here, but with reactive streams.
Related
I have a dataflow job which is written in apache beam with java. I am able run the dataflow job in GCP through this steps.
Created dataflow template from my code. Then uploading template in cloud storage.
Directly creating job from template option available in GCP->Dataflow->jobs
This flow is working fine.
I want to do same step through java app. means, I have one api when someone sends request to that api, I want to start this dataflow job through the template which I have already stored in storage.
I could see rest api is available to implement this approach. as below,
POST /v1b3/projects/project_id/locations/loc/templates:launch?gcsPath=template-location
But I didn't find any reference or samples for this. I tried the below approach
In my springboot project I added this dependency
<!-- https://mvnrepository.com/artifact/com.google.apis/google-api-services-dataflow -->
<dependency>
<groupId>com.google.apis</groupId>
<artifactId>google-api-services-dataflow</artifactId>
<version>v1b3-rev20210825-1.32.1</version>
</dependency>
and added below code in controller
public static void createJob() throws IOException {
GoogleCredential credential = GoogleCredential.fromStream(new FileInputStream("myCertKey.json")).createScoped(
java.util.Arrays.asList("https://www.googleapis.com/auth/cloud-platform"));
try{
Dataflow dataflow = new Dataflow.Builder(new LowLevelHttpRequest(), new JacksonFactory(),
credential).setApplicationName("my-job").build(); --- this gives error
//RuntimeEnvironment
RuntimeEnvironment env = new RuntimeEnvironment();
env.setBypassTempDirValidation(false);
//all my env configs added
//parameters
HashMap<String,String> params = new HashMap<>();
params.put("bigtableEmulatorPort", "-1");
params.put("gcsPath", "gs://bucket//my.json");
// all other params
LaunchTemplateParameters content = new LaunchTemplateParameters();
content.setJobName("Test-job");
content.setEnvironment(env);
content.setParameters(params);
dataflow.projects().locations().templates().launch("project-id", "location", content);
}catch (Exception e){
log.info("error occured", e);
}
}
This gives {"id":null,"message":"'boolean com.google.api.client.http.HttpTransport.isMtls()'"}
error in this line itself
Dataflow dataflow = new Dataflow.Builder(new LowLevelHttpRequest(), new JacksonFactory(),
credential).setApplicationName("my-job").build();
this is bcs, this dataflow builder expects HttpTransport as 1st argument but I passed LowLevelHttpRequest()
I am not sure is this the correct way to implement this. Can any one suggest any ideas on this? how to implement this? any examples or reference ?
Thanks a lot :)
I am working on Fabric8 unit test, now I am trying to create a CRD against KubernetesServer.
import io.fabric8.kubernetes.api.model.apiextensions.v1.CustomResourceDefinition;
public class TestCertManagerService {
#Rule
public KubernetesServer server = new KubernetesServer();
#Test
#DisplayName("Should list all CronTab custom resources")
public void testCronTabCrd() throws IOException {
// Given
//server.expect().get().withPath("/apis/stable.example.com/v1/namespaces/default/crontabs").andReturn(HttpURLConnection.HTTP_OK, ?????).once();
KubernetesClient client = server.getClient();
CustomResourceDefinition cronTabCrd = client.apiextensions().v1().customResourceDefinitions()
.load(new BufferedInputStream(new FileInputStream("src/test/resources/crontab-crd.yml")))
.get();
client.apiextensions().v1().customResourceDefinitions().create(cronTabCrd);
}
}
When I ran it, I got the following error
TestCertManagerService > testCronTabCrd FAILED
io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://localhost:60690/apis/apiextensions.k8s.io/v1/customresourcedefinitions.
at app//io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:694)
at app//io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:673)
at app//io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:626)
at app//io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:566)
at app//io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:527)
at app//io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:510)
at app//io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:136)
at app//io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:505)
at app//io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:494)
at app//io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:87)
at app//com.ibm.si.qradar.cp4s.service.certmanager.TestCertManagerService.testCronTabCrd(TestCertManagerService.java:94)
I have a few of questions:
(1) In this case, I am using v1() interface, sometimes I saw example code is using v1beta1(), what decides this version? By the way, I am using Kubernetes-client library 5.9.0
(2) In my code , I comments out this line
server.expect().get().withPath("/apis/stable.example.com/v1/namespaces/default/crontabs").andReturn(HttpURLConnection.HTTP_OK, ?????).once();
What is this statement for? In my case, I want to load a CRD, then create a CR, what is "?????" in the statement?
Any ideas for stack trace? How to fix it?
I appreciate it in advance.
From the code which you shared, it looks like you're using Fabric8 Kubernetes Mock Server in expectations mode. Expectations mode requires the user to set the REST API expectations. So the code shown below is setting some expectations from Mock Server viewpoint.
// Given
server.expect().get()
.withPath("/apis/stable.example.com/v1/namespaces/default/crontabs")
.andReturn(HttpURLConnection.HTTP_OK, getCronTabList())
.once();
These are the expectations set:
Mock Server would be requested a GET request at this URL: /apis/stable.example.com/v1/namespaces/default/crontabs . From URL we can expect a resource under stable.example.com apigroup with v1 version, default namespace and crontabs as plural.
When this URL is being hit, you're also defining response code and response body in andReturn() method. First argument is the response code (200 in this case) and second argument is the response body (a List object of CronTab which would be serialized and sent as response by mock server).
This request is only hit .once(), if KubernetesClient created by Mock Server requests this endpoint more than once; the test would fail. If you want to hit the endpoint more than once, you can use .times(..) method instead.
But in your test I see you're loading a CustomResourceDefinition from YAML and creating it which doesn't seem to match the expectations you set earlier. If you're writing a test about creating a CustomResourceDefinition, it should look like this:
#Test
#DisplayName("Should Create CronTab CRD")
void testCronTabCrd() throws IOException {
// Given
KubernetesClient client = server.getClient();
CustomResourceDefinition cronTabCrd = client.apiextensions().v1()
.customResourceDefinitions()
.load(new BufferedInputStream(new FileInputStream("src/test/resources/crontab-crd.yml")))
.get();
server.expect().post()
.withPath("/apis/apiextensions.k8s.io/v1/customresourcedefinitions")
.andReturn(HttpURLConnection.HTTP_OK, cronTabCrd)
.once();
// When
CustomResourceDefinition createdCronTabCrd = client.apiextensions().v1()
.customResourceDefinitions()
.create(cronTabCrd);
// Then
assertNotNull(createdCronTabCrd);
}
Bdw, if you don't like setting REST expectations. Fabric8 Kubernetes Mock Server also has a CRUD mode which mock real Kubernetes APIServer. You can enable it like this:
#Rule
public KubernetesServer server = new KubernetesServer(true, true);
then use it in test like this:
#Test
#DisplayName("Should Create CronTab CRD")
void testCronTabCrd() throws IOException {
// Given
KubernetesClient client = server.getClient();
CustomResourceDefinition cronTabCrd = client.apiextensions().v1()
.customResourceDefinitions()
.load(new BufferedInputStream(new FileInputStream("src/test/resources/crontab-crd.yml")))
.get();
// When
CustomResourceDefinition createdCronTabCrd = client.apiextensions().v1()
.customResourceDefinitions()
.create(cronTabCrd);
// Then
assertNotNull(createdCronTabCrd);
}
I added CustomResourceLoadAndCreateTest and CustomResourceLoadAndCreateCrudTest tests in my demo repository: https://github.com/r0haaaan/kubernetes-mockserver-demo
Im getting an error with the blocking operation in Spring Webflux. I retrieve a Mono of list of Address documents and im using this Mono list of address documents to form the street address(withStreet)as shown below :
Mono<List<Address>> docs = getAddress(id, name);
AddressResponse addrResponse = new AddressResponse.Builder().
withStreet(docs.map(doc -> doc.stream().
map(StreetAddress::map).
collect(Collectors.toList())).block()).
build();
map method :
public static StreetAddress map(Address addr) {
return new Builder().
withId(addr.getId()).
withStreet(addr.getStreetAddress()).
build();
}
When i execute the above code, it throws a "block()/blockFirst()/blockLast() are blocking, which is not supported in thread reactor-http-nio-2". Could you please suggest how to fix. i want to retrieve AddressResponse without blocking it. This response will be further used in the code in Response Entity as shown below :
return Mono.just(ResponseEntity
.status(HttpStatus.OK)
.body(addrResponse)));
The problem is you try to mix reactive and imperative code.
Instead, just map it in the reactive pipeline:
Mono<AddressResponse> response = docs.map(addresses->{
return new AddressResponse.Builder()
.withStreet(addresses -> addresses.stream()
.map(StreetAddress::map)
.collect(Collectors.toList()))
.build();
})
Then you can return it as is, or map it into a Mono> type, apply the same method then above.
We are trying to call a REST API using HTTP OutboundGateway using DSL. We are able to make the call using both GET and POST and getting the response as expected. However we couldnt figure a way to pass http headers while making this call using DSL. There are quite a lot of articles about XML approach but couldnt find out documentation with DSL
return IntegrationFlows.from("FileContentChannel")
.handle(Http.outboundGateway("http://host:port/paymentinfo/")
.charset("UTF-8")
.httpMethod(HttpMethod.GET)
.headerMapper(headers)
.expectedResponseType(String.class))
.channel(MessageChannels.queue("APIResponseChannel"))
.get();
We tried with DefaultHttpHeaderMapper as well but it didnt work. Can you please guide us on this?
Thanks to Gary..it worked with this
Update1
return IntegrationFlows.from("FileContentChannel")
.handle(Http.outboundGateway("http://host:port/paymentinfo/")
.charset("UTF-8")
.httpMethod(HttpMethod.GET)
.mappedRequestHeaders("pay*")
.headerMapper(headerMapper())
.expectedResponseType(String.class))
.channel(MessageChannels.queue("APIResponseChannel"))
.get();
#Bean
HeaderMapper headerMapper() {
DefaultHttpHeaderMapper headerMapper = new DefaultHttpHeaderMapper();
String[] headerNames = {"payment-hdr1","payment-hdr2"};
headerMapper.setOutboundHeaderNames(headerNames);
headerMapper.setUserDefinedHeaderPrefix("");
return headerMapper;
}
return IntegrationFlows.from("FileContentChannel")
.enrichHeaders(h -> h.header("foo1", "bar")
.header("foo2", "baz"))
.handle(Http.outboundGateway("http://host:port/paymentinfo/")
.charset("UTF-8")
.httpMethod(HttpMethod.GET)
.mappedRequestHeaders("foo*")
.expectedResponseType(String.class))
.channel(MessageChannels.queue("APIResponseChannel"))
.get();
Custom headers will (currently) get an X- prefix.
To avoid that use a customized DefaultHeaderMapper to map the required header patterns and use a userDefinedHeaderPrefix of "".
I am trying to remove an alias mapping for an index in ES using jest.
Here is what I have tried :
// create Jest Client.
JestClient client = factory.getObject();
// create RemoveAliasMapping Object.
RemoveAliasMapping removeAliasMapping = new RemoveAliasMapping.Builder("oldIndex", "alias").build();
After creating the removeAliasMapping object, I couldn't find a way to execute it.
If I use the api : client.execute(removeAliasMapping), it says : The method execute(Action<T>) in the type JestClient is not applicable for the arguments (RemoveAliasMapping)
Also, I couldn't find any other api exposed to execute AliasMapping.
Can anyone help me out with this here? If possible, please put an example too.
Try this:
ModifyAliases modifyAliases = new ModifyAliases.Builder(new RemoveAliasMapping.Builder("oldIndex", "alias").build()).build();
JestResult result = client.execute(modifyAliases);