Elasticsearch java RestHighLevelClient "Unable to parse response body" IllegalArgumentException: Required [index] - parsing

I'm dealing with a problem when creating an index using the java RestHighLevelClient in Elasticsearch and my CreateIndexResponse object is in consequence null.
I am actually able to create the index, which I can confirm later querying it, but when I create the index, I get this exception. Here my code:
`CreateIndexRequest request = new CreateIndexRequest("myindex");
CreateIndexResponse createIndexResponse = client.indices().create(request);`
Elasticsearch returns the message of success with:
`HTTP 200 Success
{
"acknowledged": true,
"shards_acknowledged": true
}`
And I am actually able to retrieve the index later with a GET call, but when the RestHighLevelClient tries to parse the response, using the following internal call:
//Type of the response converter: CheckedFunction<Req, Request, IOException> requestConverter
responseConverter.apply(response);
The following exception happens:
java.io.IOException: Unable to parse response body for
Response{requestLine=PUT /myindex?master_timeout=30s&timeout=30s HTTP/1.1,
host=http://localhost:9200, response=HTTP/1.1 200 OK}
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:507)
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:474)
at org.elasticsearch.client.IndicesClient.create(IndicesClient.java:77)
at hello.client.HelloClient.createSynch(HelloClient.java:84)
at hello.main.Main.main(Main.java:25)
Caused by: java.lang.IllegalArgumentException: Required [index]
So basically what this is saying is that the following response cannot be parsed, but for me it looks pretty parsable:
Response{requestLine=PUT /myindex?master_timeout=30s&timeout=30s HTTP/1.1,
host=http://localhost:9200, response=HTTP/1.1 200 OK}
Why does it tell me that the index is missing? Is it that I'm using wrongly the java client? This is the version:
<dependencies>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>6.2.1</version>
</dependency>
</dependencies>`
Thanks in advance for the help!

You need to either update your version dependency or add compatibility headers (https://docs.spring.io/spring-data/elasticsearch/docs/current/reference/html/) as for my case even latest version of Spring-data-elastic search doesn't support version 8+ of Elastic Search. Had to configure my client like this:
#Configuration
#EnableElasticsearchRepositories(basePackages = "*")
public class ElasticsearchClientConfig {
#Value("${elasticsearch.host}")
private String host;
#Value("${elasticsearch.port}")
private int port;
#Value("${elasticsearch.protocol}")
private String protocol;
#Value("${elasticsearch.username}")
private String userName;
#Value("${elasticsearch.password}")
private String password;
#Bean(destroyMethod = "close")
public RestHighLevelClient restClient() {
final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(userName, password));
RestClientBuilder builder = RestClient.builder(new HttpHost(host, port, protocol))
.setHttpClientConfigCallback(httpClientBuilder -> httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider))
.setDefaultHeaders(compatibilityHeaders());
return new RestHighLevelClient(builder);
}
private Header[] compatibilityHeaders() {
return new Header[]{new BasicHeader(HttpHeaders.ACCEPT, "application/vnd.elasticsearch+json;compatible-with=7"), new BasicHeader(HttpHeaders.CONTENT_TYPE, "application/vnd.elasticsearch+json;compatible-with=7")};
}
}

Related

JNDI: ApacheDS: INVALID_CREDENTIALS: DIGEST-MD5: digest response format violation. Mismatched URI: ldap/localhost; expecting: ldap/ldap.example.com

I am trying to authenticate user using JNDI, security level as SASL. Following is my sample code.
import java.util.Hashtable;
import javax.naming.Context;
import javax.naming.NamingException;
import javax.naming.directory.DirContext;
import javax.naming.directory.InitialDirContext;
public class Test {
private static final String CONTEXT_FACTORY = "com.sun.jndi.ldap.LdapCtxFactory";
private static final String PROVIDER_URL = "ldap://localhost:10389";
private static final String SECURITY_AUTHENTICATION = "DIGEST-MD5";
public static void main(String[] args) throws NamingException {
Hashtable<String, String> env = new Hashtable<String, String>(11);
env.put(Context.INITIAL_CONTEXT_FACTORY, CONTEXT_FACTORY);
env.put(Context.PROVIDER_URL, PROVIDER_URL);
env.put(Context.SECURITY_AUTHENTICATION, SECURITY_AUTHENTICATION);
env.put(Context.SECURITY_PRINCIPAL,
"cn=Krishna,ou=people,dc=example,dc=com");
env.put(Context.SECURITY_CREDENTIALS, "password123");
try {
DirContext ctx = new InitialDirContext(env);
System.out.println("Authentication Successful");
ctx.close();
} catch (NamingException e) {
System.out.println("Authentication Failed");
e.printStackTrace();
}
}
}
I encrypted the password using MD5 algorithm in directory. When I tried to run above program, I am getting following error.
Authentication Failed
javax.naming.AuthenticationException: [LDAP: error code 49 - INVALID_CREDENTIALS: DIGEST-MD5: digest response format violation. Mismatched URI: ldap/localhost; expecting: ldap/ldap.example.com]
at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3135)
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:3081)
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2883)
at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2797)
at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:319)
at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:192)
at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:210)
at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:153)
at com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:83)
at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)
at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:313)
at javax.naming.InitialContext.init(InitialContext.java:244)
at javax.naming.InitialContext.<init>(InitialContext.java:216)
at javax.naming.directory.InitialDirContext.<init>(InitialDirContext.java:101)
at jndi_tutorial.Test.main(Test.java:26)
But when i tried to authenticate using simple mechanism (SECURITY_AUTHENTICATION = "simple"), My authentication is success. Is there any configurations I am missing?
Check if your LDAP server supports DIGEST-MD5 SASL mechanism.
DirContext ctx = new InitialDirContext();
Attributes attrs = ctx.getAttributes("ldap://<HSOT>:<PORT>", new String[]{"supportedSASLMechanisms"});
Check if the passwords are in fact stored as MD5 digest/hash in the LDAP server.
Connect to the LDAP sever with an LDPAP browser like Apache Dir Studio and check the password attribute. It will be prefixed with hash mechanism used.

Spring Security 4 issue using Query Method

I am using the latest spring security 4 version and it introduces a new feature to use the logged in user details directly in the query method using expression language. Here is my spring data repository code:
public interface UserRepository extends JpaRepository<User, Long> {
#Query("select username from User u where u.username = ?#{ principal?.username }")
User findByUsername(String username);
}
In the above code, I have an entity User as below:
#Entity
#Table(name = "users")
public class User {
#Id
#Column(name = "username", nullable = false, unique = true)
private String username;
#Column(name = "password", nullable = false)
#NotNull
private String password;
#Column(name = "enabled", nullable = false)
#NotNull
private Boolean enabled;
#Column(name = "role", nullable = false)
#Enumerated(EnumType.STRING)
private Role role;
//getters and setters
Also I have this entry for enabling this feature:
#Bean
public SecurityEvaluationContextExtension securityEvaluationContextExtension() {
return new SecurityEvaluationContextExtension();
}
When I run the application, I get the error:
Caused by: org.springframework.dao.InvalidDataAccessApiUsageException: Authentication object cannot be null; nested exception is java.lang.IllegalArgumentException: Authentication object cannot be null
at org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:381)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:223)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.translateExceptionIfPossible(AbstractEntityManagerFactoryBean.java:417)
Caused by: java.lang.IllegalArgumentException: Authentication object cannot be null
at org.springframework.security.access.expression.SecurityExpressionRoot.<init>(SecurityExpressionRoot.java:46)
at org.springframework.security.data.repository.query.SecurityEvaluationContextExtension$1.<init>(SecurityEvaluationContextExtension.java:113)
at org.springframework.security.data.repository.query.SecurityEvaluationContextExtension.getRootObject(SecurityEvaluationContextExtension.java:113)
at org.springframework.data.repository.query.ExtensionAwareEvaluationContextProvider$EvaluationContextExtensionAdapter.<init>(ExtensionAwareEvaluationContextProvider.java:463)
at org.springframework.data.repository.query.ExtensionAwareEvaluationContextProvider.toAdapters(ExtensionAwareEvaluationContextProvider.java:210)
at org.springframework.data.repository.query.ExtensionAwareEvaluationContextProvider.access$000(ExtensionAwareEvaluationContextProvider.java:58)
What could be the issue. Here I am posting to check if there is any issue in using the query method. Can i use like principal.username, is that correct?
Update: When I removed the #Query from repository it works fine. That means its problem with the new spring security 4 using principal.username. Is there anything wrong in this expression?
#Query("select username from User u where u.username = ?#{ principal?.username }")
Pls try this custom class :-
class SecurityEvaluationContextExtension extends EvaluationContextExtensionSupport {
#Override
public String getExtensionId() {
return "Security";
}
#Override
public SecurityExpressionRoot getRootObject() {
Authentication authentication = SecurityContextHolder.getContext().getAuthentication();
return new SecurityExpressionRoot(authentication){};
}
}
Not sure whether you have solved the issue in the meantime, but I noticed that your query should look like:
select u from User u where u.username = ?#{ principal }
assuming your principal object is the plain username String.
If you created your own SecurityEvaluationContextExtension class, and did not implement getAuthentication() method, you might be getting this exception.
In this link, you can see original SecurityEvaluationContextExtension.java file, that implements all necessary methods.
So, you don't need to implement this class on your own. Instead, you can add below dependency to your pom file to have the original one;
<!-- https://mvnrepository.com/artifact/org.springframework.security/spring-security-data -->
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-data</artifactId>
<version>4.2.1.RELEASE</version>
</dependency>
If you are using any other dependency manager rather than maven, you can goto related maven repo and get the definition that you want.
I hope this helps.

ServiceStack authentication request fails

I am trying to set up authentication with my ServiceStack service by following this tutorial.
My service is decorated with the [Authenticate] attribute.
My AppHost looks like this:
public class TestAppHost : AppHostHttpListenerBase
{
public TestAppHost() : base("TestService", typeof(TestService).Assembly) { }
public static void ConfigureAppHost(IAppHost host, Container container)
{
try
{
// Set JSON web services to return idiomatic JSON camelCase properties.
ServiceStack.Text.JsConfig.EmitCamelCaseNames = true;
// Configure the IOC container
IoC.Configure(container);
// Configure ServiceStack authentication to use our custom authentication providers.
var appSettings = new AppSettings();
host.Plugins.Add(new AuthFeature(() =>
new AuthUserSession(), // use ServiceStack's session class but fill it with our own data using our own auth service provider
new IAuthProvider[] {
new UserCredentialsAuthProvider(appSettings)
}));
}
}
where UserCredentialsAuthProvider is my custom credentials provider:
public class UserCredentialsAuthProvider : CredentialsAuthProvider
{
public override bool TryAuthenticate(IServiceBase authService, string userName, string password)
{
try
{
// Authenticate the user.
var userRepo = authService.TryResolve<IUserRepository>();
var user = userRepo.Authenticate(userName, password);
// Populate session properties.
var session = authService.GetSession();
session.IsAuthenticated = true;
session.CreatedAt = DateTime.UtcNow;
session.DisplayName = user.FullName;
session.UserAuthName = session.UserName = user.Username;
session.UserAuthId = user.ID.ToString();
}
catch (Exception ex)
{
// ... Log exception ...
return false;
}
return true;
}
}
In my user tests I initialize and start my TestAppHost on http://127.0.0.1:8888, then use JsonServiceClient to authenticate itself to the service like so:
var client = new JsonServiceClient("http://127.0.0.1:8888/")
var response = client.Send<AuthResponse>(new Auth
{
provider = UserCredentialsAuthProvider.Name,
UserName = username,
Password = password,
RememberMe = true
});
But getting the following exception:
The remote server returned an error: (400) Bad Request.
at System.Net.HttpWebRequest.GetResponse()
at ServiceStack.ServiceClient.Web.ServiceClientBase.Send[TResponse](Object request)...
The ServiceStack.ServiceInterface.Auth.Auth request contains the correct username and passsword, and the request is being posted to:
http://127.0.0.1:8888/json/syncreply/Auth
I am not sure why the URL is not /json/auth/credentials or what I might be doing wrong. Any suggestions?
UPDATE
Tracing the chain of events up the stack I found the following:
JsonDataContractSerializer.SerializeToStream correctly serializes the Auth request into Json. However, the System.Net.HttpRequestStream passed to JsonDataContractDeserializer by EndpointHandlerBase has a stream of the correct length that is filled with nulls (zero bytes). As a result, the request object passed to CredentialsAuthProvider.Authenticate has nulls in all its properties.
How can the HTTP stream get stripped of its data?
Got it!!!
The problem was the following pre-request filter that I added for logging purposes in TestAppHost.Configure:
PreRequestFilters.Add((httpReq, httpRes) =>
{
LastRequestBody = httpReq.GetRawBody();
});
as seen here.
When the GetRawBody() method reads the request InputStream it leaves it in the EOS state, and all subsequent read attempts return nothing.
So obviously GetRawBody() can only be safely used with buffered streams, but unfortunately it quietly causes a very nasty bug instead of throwing an exception when used with a non-buffered stream.

Posting multipart form data to seam+RESTeasy fails marshalling to InputStream

I'm trying to post image data to a seam+RESTeasy endpoint and I'm getting a very cryptic error during JBoss startup. The HTTP request I'm sending has a content-type of multipart/form-data which has a single image/jpeg part with name "attachment". My service method looks like this:
#POST
#Path("uploadSymptomsImage/{appointmentGUID}")
#Consumes(MediaType.MULTIPART_FORM_DATA)
#Produces("application/json")
public String uploadSymptomsImage( #FormParam("attachment") InputStream fileInputStream,
#PathParam("appointmentGUID") String strAppointmentGUID )
{ ...
The error that I get is during startup:
Caused by: java.lang.RuntimeException: Unable to find a constructor that takes a String param or a valueOf() or fromString() method for javax.ws.rs.FormParam("attachment") on public java.lang.String com....AppointmentRestService.uploadSymptomsImage(java.io.InputStream,java.lang.String) for basetype: java.io.InputStream
at org.jboss.resteasy.core.StringParameterInjector.initialize(StringParameterInjector.java:206) [:]
at org.jboss.resteasy.core.StringParameterInjector.<init>(StringParameterInjector.java:57) [:]
at org.jboss.resteasy.core.FormParamInjector.<init>(FormParamInjector.java:22) [:]
My understanding was that media types could be automatically marshalled to InputStream. I've also tried java.io.File, java.io.Reader - both with same error. When I replace with byte[] or String I get a zero length array, or null as the parameter value.
How would you go about debugging this? Also, is it possible to access the raw request or pre-marshalled values?
Any suggestions here would be greatly appreciated.
You should retrieve the contents using MultipartFormDataInput. See the following example:
#POST
#Path("uploadSymptomsImage/{appointmentGUID}")
#Consumes(MediaType.MULTIPART_FORM_DATA)
#Produces("application/json")
public String uploadSymptomsImage(#PathParam("appointmentGUID") String strAppointmentGUID,
MultipartFormDataInput formData) {
Map<String, List<InputPart>> formDataMap = formData.getFormDataMap();
List<InputPart> attachments = formDataMap.get("attachment");
for(InputPart attachment : attachments) {
String fileName = extractFilename(attachment);
if(fileName.isEmpty()) continue;
InputStream in = attachment.getBody(new GenericType<InputStream>() {});
// Interact with stream
}
// Respond
}
The extractFilename method is a helper method I wrote:
private static String extractFilename(final InputPart attachment) {
Preconditions.checkNotNull(attachment);
MultivaluedMap<String, String> headers = attachment.getHeaders();
String contentDispositionHeader = headers.getFirst("Content-Disposition");
Preconditions.checkNotNull(contentDispositionHeader);
for(String headerPart : contentDispositionHeader.split(";(\\s)+")) {
String[] split = headerPart.split("=");
if(split.length == 2 && split[0].equalsIgnoreCase("filename")) {
return split[1].replace("\"", "");
}
}
return null;
}

Setting timeout for new URL(...).text in Groovy/Grails

I use the following Groovy snippet to obtain the plain-text representation of an HTML-page in a Grails application:
String str = new URL("http://www.example.com/some/path")?.text?.decodeHTML()
Now I want to alter the code so that the request will timeout after 5 seconds (resulting instr == null). What is the easiest and most Groovy way to achieve that?
I checked source code of groovy 2.1.8, below code is available:
'http://www.google.com'.toURL().getText([connectTimeout: 2000, readTimeout: 3000])
The logic to process configuration map is located in method org.codehaus.groovy.runtime.ResourceGroovyMethods#configuredInputStream
private static InputStream configuredInputStream(Map parameters, URL url) throws IOException {
final URLConnection connection = url.openConnection();
if (parameters != null) {
if (parameters.containsKey("connectTimeout")) {
connection.setConnectTimeout(DefaultGroovyMethods.asType(parameters.get("connectTimeout"), Integer.class));
}
if (parameters.containsKey("readTimeout")) {
connection.setReadTimeout(DefaultGroovyMethods.asType(parameters.get("readTimeout"), Integer.class));
}
if (parameters.containsKey("useCaches")) {
connection.setUseCaches(DefaultGroovyMethods.asType(parameters.get("useCaches"), Boolean.class));
}
if (parameters.containsKey("allowUserInteraction")) {
connection.setAllowUserInteraction(DefaultGroovyMethods.asType(parameters.get("allowUserInteraction"), Boolean.class));
}
if (parameters.containsKey("requestProperties")) {
#SuppressWarnings("unchecked")
Map<String, String> properties = (Map<String, String>) parameters.get("requestProperties");
for (Map.Entry<String, String> entry : properties.entrySet()) {
connection.setRequestProperty(entry.getKey(), entry.getValue());
}
}
}
return connection.getInputStream();
}
You'd have to do it the old way, getting a URLConnection, setting the timeout on that object, then reading in the data through a Reader
This would be a good thing to add to Groovy though (imho), as it's something I could see myself needing at some point ;-)
Maybe suggest it as a feature request on the JIRA?
I've added it as a RFE on the Groovy JIRA
https://issues.apache.org/jira/browse/GROOVY-3921
So hopefully we'll see it in a future version of Groovy...

Resources