In JAVA code, not running in the Google environment, how does one use a trained translation model? - translation

I seem to be missing something obvious, I think. We've been using the Google Translation API for a while now, and now we want to "upgrade" to a custom trained model instead of the default nmt.
We've uploaded our texts, trained it, and now have a model. In the predict tab on the Google console, it works great. so, now what?
This is the code that we use today:
translate = TranslateOptions
.newBuilder()
.setCredentials(ServiceAccountCredentials.fromStream(googleCredentials))
.build()
.getService();
translate.translate(
text,
TranslateOption.sourceLanguage(fromLng),
TranslateOption.targetLanguage(toLng),
TranslateOption.model(model));
where model is "nmt" (or "base")... should I just be able to drop in my newly trained model code that was created when the training finished? When I try, it comes back with a 400 error and the message:
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Invalid Value",
"reason" : "invalid"
} ],
"message" : "Invalid Value"
Trying different code as documented here: https://cloud.google.com/translate/docs/quickstart-client-libraries-v3
yields other errors like: "INFO: Failed to detect whether we are running on Google Compute Engine."
Where am I going wrong?

here we go... for the next person looking to do this:
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-automl</artifactId>
<version>0.97.0-beta</version>
</dependency>
code:
private PredictionServiceClient predictionClient;
private ModelName modelName;
public GoogleTranslationServiceTrained(final byte[] googleCredentials) throws IOException {
super();
PredictionServiceSettings settings = PredictionServiceSettings
.newBuilder()
.setCredentialsProvider(new CredentialsProvider() {
#Override
public Credentials getCredentials() throws IOException {
return ServiceAccountCredentials.fromStream(new ByteArrayInputStream(googleCredentials));
}
}).build();
// Instantiate client for prediction service.
predictionClient = PredictionServiceClient.create(settings);
// Get the full path of the model.
modelName = ModelName.of("xxxx", "us-central1", "yyy");
}
public String getRemoteTranslate(String text) {
TextSnippet textSnippet = TextSnippet.newBuilder().setContent(text).build();
// Set the payload by giving the content of the file.
ExamplePayload payload = ExamplePayload.newBuilder().setTextSnippet(textSnippet).build();
// Additional parameters that can be provided for prediction
Map<String, String> params = new HashMap<>();
PredictResponse response = predictionClient.predict(modelName, payload, params);
TextSnippet translatedContent = response.getPayload(0).getTranslation().getTranslatedContent();
return StringEscapeUtils.unescapeHtml4(translatedContent.getContent());
}

Related

Need to Pass the 'value' to my #CustomPreAuthorize annotation and use it in #PreAuthorize("hasAuthority(#myservice.check(#value))")

Currently I am writing my own custom #PreAuthorize annotation.
My case is as follows,
I am running my authorization KeyCloak server that holds the user details, roles and permission
After Validation, I have stored the permission details in GrantedAuthority as follows "{rsname}:GET", "{rsname}:POST" ...
KeyCloak JWT permission structure:
"authorization": {
"permissions": [
{
"scopes": [
"GET",
"DELETE",
"POST"
],
"rsid": "6ae9895f-3766-464f-82c4-44f598ec2a93",
"rsname": "record"
}
]
}
while using #PreAuthorize annotation in controller instead of hardcoding the resource name and scopes, we have to generalize it by getting the details from application.property we have achieved it as follows,
application.property:
auth:
data:
name1: record
name2: device
Property Detail Component class:
#ConfigurationProperties(prefix = "auth")
#Component
public class SecurityProperty {
private Map<String, String> data;
....
}
Controller:
#RequestMapping (method = RequestMethod.GET,value = "/api/records",
produces = {"application/json"})
#PreAuthorize ("hasAuthority (#securityProperty.getData(). get('name1') "
+ "+ ': GET')")
ResponseEntity<List<SomeDTO>> getRecords() {
...Some Logic
}
#RequestMapping(method = RequestMethod.GET,value = "/api/devices",
produces = { "application/json" })
#PreAuthorize("hasAuthority(#securityProperty.getResources().get('name2') "
+ "+ ':GET')")
ResponseEntity<List<SomeDTO>> getDevices() {
...Some Logic
}
So far this is working fine. Since we are creating big project we don't want to write this lengthy #PreAuthorize(XXXX) annotation so decided to create custom annotation that uses the #PreAuthorize.
We have created #CustomPreAuthorize as below
#Retention(RUNTIME)
#Target({ElementType.METHOD, ElementType.TYPE})
#PreAuthorize("hasAuthority(#securityProperty.getResources().get(#resource)"
+ ".concat(':GET'))")
public #interface CustomPreAuthorize {
String resource();
}
And used this in controller
#RequestMapping (method = RequestMethod.GET,value = "/api/devices",
produces = {"application/json"})
#CustomPreAuthorize (resource = "name2")
ResponseEntity<List<SomeDTO>> getDevices() {
...Some Logic
}
Issue:
When I used like this when the API is called I am getting the following error
Failed to evaluate expression 'hasAuthority(#securityProperty.getResources().get(#resource).concat(':GET'))"
So far what I understood is like the resource and scope are not getting recognized in the #PreAuthorize annotation level. Is it possible to read the values like this or is there any alternatives available?
Since there is no reply yet for the required fix, for now we have fixed this by adding the Aspect for the annotation and proceed with manual authority check using the SecurityContextHolder.
#Before("#annotation(CustomPreAuthorize)")
void annotationPointcut() {
//business logic
}
We will check for the other solutions actively and post if we achieve it in more better way.

Vaadin REST validation response

When a REST endpoint returns an validation error I want to reflect the validation message in the corresponding vaadin input field. In this example I would like to have an error message near to the password input field. So the user is able to fix the error easily.
#Route("create-customer")
public class CreateCustomerView extends Div {
public CreateCustomerView(RestClientService restClientService) {
TextField firstName = new TextField("First name");
DatePicker birthDay = new DatePicker("Birthday");
PasswordField password = new PasswordField("Password");
Button submit = new Button("Submit", (ComponentEventListener<ClickEvent<Button>>) event -> restClientService.createCustomer(
new RegisterCustomerRequest(firstName.getValue(), birthDay.getValue(), password.getValue())));
FormLayout formLayout = new FormLayout();
formLayout.add(firstName, password, submit);
add(formLayout);
}
}
My repsonse is very standard and verbose.
{
"timestamp": "2022-09-17T12:39:55.642883700",
"status": 400,
"error": "ConstraintViolationException",
"message": "Some arguments are not valid.",
"fieldErrors": [
{
"field": "register.registerCustomerCommand.password",
"error": "com.example.application.service.RegisterCustomerService register.registerCustomerCommand.password: # a digit must occur at least once\n# a lower case letter must etc. etc."
}
],
"path": "/v1/customers/"
}
Is it possible with vaadin out of the box? Binder seems to be good for client side validation errors but my errors depend on the server side validation on the request.
Solved it like this:
introduced ServersideValidationException that is thrown if response from server contains the error "ConstraintViolationException". In this case I deserialize the content to an Error that contains List of fieldErrors with path and errorMessage and pass it to the Exception.
map these fieldErrors trough the binder to the errorMessage of a Component.
// button action
private void validateAndSave() {
if (binder.isValid()) {
try {
binder.writeBean(registerCustomerRequest);
final CustomerResponse customer = restClientService.createCustomer(registerCustomerRequest);
} catch (ValidationException e) {
throw new ApplicationException(e);
} catch (ServersideValidationException e) {
populateErrorsToForm(e);
}
}
}
private void populateErrorsToForm(ServersideValidationException e) {
e.getFieldErrors().forEach(fieldError -> {
final String fieldName = StringUtils.substringAfterLast(fieldError.getPath(), ".");
final var binding = binder.getBinding(fieldName).orElseThrow(fieldNotFoundInForm(fieldName));
BeanWrapper wrapper = PropertyAccessorFactory.forBeanPropertyAccess(binding.getField());
wrapper.setAutoGrowNestedPaths(true);
wrapper.setPropertyValue("invalid", true);
wrapper.setPropertyValue("errorMessage", fieldError.getErrorMessage());
});
}
Sidenote: Binder is not required for setting the error but it provides handy method binder.getBinding(fieldName). Does not support nested errors yet it is feasible. It is ugly.
As long as you can get the validation answer quickly enough, you can just use Binder since it doesn't care about the origin of the validation result. If the response is slow, then you might have to validate asynchronously, but then you're on your own without direct help from Binder.

Springfox: Alternate type rules are not being applied

My #EnableSwagger2 annotated class contains the following method:
#Bean
public Docket myServiceApi() {
return new Docket(DocumentationType.SWAGGER_2).groupName("My Service API").apiInfo(apiInfo()).select()
.paths(PathSelectors.regex("/api.*")).build()
.alternateTypeRules(
newRule(
typeResolver.resolve(Map.class, String.class, Object.class),
typeResolver.resolve(InputExample.class)
)
)
;
}
Where InputExample is a class that contains many different properties annotated with #ApiModelProperty.
The method in my REST controller looks like this:
#ApiOperation(
value = "Do stuff",
consumes = MediaType.APPLICATION_JSON_VALUE,
produces = MediaType.APPLICATION_JSON_VALUE,
response = SomeOutput.class
)
#RequestMapping(
value = "/api/v1/stuff",
method = RequestMethod.POST,
consumes = {MediaType.APPLICATION_JSON_VALUE},
produces = {MediaType.APPLICATION_JSON_VALUE}
)
#ApiResponses(
value = {
#ApiResponse(code = 200, message = "Service execution successful"),
#ApiResponse(code = 400, message = "Bad input data"),
#ApiResponse(code = 500, message = "An internal server error occurred"),
#ApiResponse(code = 503, message = "The service is currently unavailable")
}
)
public ResponseEntity<SomeOutput> doServiceStuff(
HttpServletRequest request,
#RequestBody Map<String, Object> inputContent
) throws
ValidationException,
ServiceUnavailableException,
IOException,
WorkflowDocumentProcessingException
{
...
}
Sadly, when I run my Service and open my endpoint on Swagger UI, all I see is:
What could this be caused by? How can I debug this?
P.S.: The rest of the #EnableSwagger2 - class does work.
There already seems to be an internal rule with the original type Map<String, Object> overriding whatever the developer adds to .alternateTypeRules().
The only way I could find to fix this is to create a class MyInputMap extends Map<String, Object> and use it in all endpoints in question, while also adjusting the type rule to:
newRule(
typeResolver.resolve(MyInputMap.class),
typeResolver.resolve(InputExample.class)
)

Pact for MessageQueue's : Sample Provider test in case of MessageQueues

I have written sample Pact test for MessageProvider by referring to sample example given in pact repo. Following is the consumer test which is generating the PACT json file for the message expected from Provider.
In case of API, to verify the PACT, I was able to do this using "pact-jvm-provider-maven" plugin. In this case the PACT is verified against the actual hosted API service of Provider.
My Question is, how in case of MessageQueue, PACT will be verified? Will a mock Queue gets created? or I need to publish a message to actual Queue and need to verify the PACT message against this message published to Queue.
Can someone explain how exactly it works?
Also please point me to sample code (example test) to be written at provider end to verify the message of MessageQueue.
Sample message (Consumer) test :
public class Inbound_Receiving_OpenMessageTest {
private byte[] receivingOpenLoadDetailsMessage;
#Rule
public MessagePactProviderRule mockProvider = new MessagePactProviderRule(this);
#Pact(provider = Configuration.ReceivingProviderOpen, consumer = Configuration.InboundConsumer)
public MessagePact createPact(MessagePactBuilder builder) {
PactDslJsonBody body = (PactDslJsonBody) new PactDslJsonBody()
.stringType("_id")
.object("delivery")
.stringType("deliveryNumber")
.closeObject()
.array("state")
.object()
.stringType("changeTime")
.stringValue("status", "OPEN")
.stringType("changeUser")
.closeObject()
.closeArray();
Map<String, String> metadata = new HashMap<String, String>();
metadata.put("contentType", "application/json");
return builder
.given("Receiving(Open) App State")
.expectsToReceive("Receiving Open Load details Test")
.withMetadata(metadata)
.withContent(body)
.toPact();
}
#Test
#PactVerification({Configuration.ReceivingProviderOpen, "Receiving(Open) App State"})
public void test() throws Exception {
Assert.assertNotNull(new String(receivingOpenLoadDetailsMessage));
LoadDetails openLoadDetails = null;
Gson gson = new GsonBuilder().create();
String entity = new String(receivingOpenLoadDetailsMessage);
openLoadDetails = gson.fromJson(entity, LoadDetails.class);
if(openLoadDetails.getDelivery().getDeliveryNumber() == null ||
openLoadDetails.getState().get(0).getChangeUser() == null ||
openLoadDetails.getState().get(0).getChangeTime() == null ||
openLoadDetails.getState().get(0).getStatus() == null){
Assert.fail("Either one of the field 'deliveryNumber' or 'changeTime' or 'status' or 'changeUser' is NULL");
}
}
public void setMessage(byte[] messageContents) {
receivingOpenLoadDetailsMessage = messageContents;
}
}
This blog post explains it in more detail.
Essentially, the idea is that if you can verify that the code that puts the message onto the queue conforms to the contract (the provider), and the code that handles the message from the queue also conforms to the contract (the consumer), you don't actually need a message queue to verify the contract.

Better way to handle common headers and root?

Is there a better way to set bearer like a global config rather than setting it each time like this:
restClient.setBearerAuth(TokenStore.getInstance().getLocalToken());
The same for root url, is there a global config rather than setting it like this:
String root= Application.getInstance().getApplicationContext().getResources().getString(R.string.whiteLabelApiBaseHost)
restClient.setRootUrl(root);
In retrofit, there is something like this:
Retrofit retrofit = new Retrofit.Builder()
.baseUrl(Application.getInstance().getApplicationContext()
.getResources().getString(R.string.whiteLabelApiBaseHost))
Any idea?
To set root url you can use this method, substituting the string with a constant
#Rest(rootUrl = "http://company.com/ajax/services", converters = { MappingJackson2HttpMessageConverter.class }, interceptors = MyAuthInterceptor.class)
public interface MyRestClient {
#Get("/events")
EventList getEvents();
}
Note that we set an interceptor in the arguments of the #Rest annotation.
So create a class like this:
#EBean(scope = Scope.Singleton)
public class MyAuthInterceptor implements ClientHttpRequestInterceptor {
#Bean
MyAuthStore authStore;
public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException {
HttpHeaders headers = request.getHeaders();
HttpAuthentication auth = new HttpBasicAuthentication(authStore.getUsername(), authStore.getPassword());
headers.setAuthorization(auth);
return execution.execute(request, body);
}
}
Now before executing request MyAuthInterceptor.intercept() is called and you can set your authentication data as you prefer
In your main build.gradle file you can add inside android element
productFlavors {
development {
buildConfigField "String", "SERVICE_URL_BASE", "\"dev.xxx.com/rest\""
}
test {
buildConfigField "String", "SERVICE_URL_BASE", "\"test.xxx.com/rest\""
}
production {
buildConfigField "String", "SERVICE_URL_BASE", "\"production.xxx.com/rest\""
}
}
Then in your #Rest annotation you can use this code to get current flavor value:
#Rest(rootUrl = "https://" + BuildConfig.SERVICE_URL_BASE)
Now you can select what build variant to use (variant = flavor + buildType) to use desired value. To select variant you can use corresponding view, it should be present on the left of android studio.
This technique is useful to avoid creating flavor's package tree only to use different variabiles

Resources