Please can you help me writing junits for below method,
public Map<String, List<Entity>> getEntitiesUsingAncestorKey(Key ancestorKey) {
QueryResults<Entity> queryResults;
StringBuffer queryString = new StringBuffer();
queryString.append("SELECT * WHERE __key__ HAS ANCESTOR #ancestorKey");
Query<Entity> query = Query.newGqlQueryBuilder(Query.ResultType.ENTITY, queryString.toString())
.setBinding("ancestorKey", ancestorKey)
.build();
queryResults = datastore.run(query);
return StreamSupport.stream(
Spliterators.spliteratorUnknownSize(queryResults, Spliterator.ORDERED), false)
.collect(Collectors.groupingBy(entity -> entity.getKey().getKind()));
}
How to mock QueryResults..?
Related
Hello I have a common function which looks like below,
public async Task<SPResponse> ExecuteAsync(string spName, DynamicParameters p)
{
SPResponse response = new SPResponse();
using (SqlConnection conn = new SqlConnection(_connStr))
{
conn.Open();
using (SqlTransaction transaction = conn.BeginTransaction(IsolationLevel.ReadCommitted))
{
try
{
p.Add("#SP_MESSAGE", dbType: DbType.String, direction: ParameterDirection.Output, size: 4000);
p.Add("#RETURNSTATUS", dbType: DbType.Int32, direction: ParameterDirection.ReturnValue);
await conn.ExecuteAsync(sql: spName, param: p, commandType: CommandType.StoredProcedure, transaction: transaction);
response.ReturnMessage = p.Get<string>("#SP_MESSAGE");
response.ReturnStatus = Convert.ToString(p.Get<int>("#RETURNSTATUS"));
if (response.ReturnStatus == "0")
{
response.Ref1 = Convert.ToString(p.Get<int>("#SP_ID"));
transaction.Commit();
}
else
{
transaction.Rollback();
}
}
catch (Exception ex)
{
Utils.Logger.Instance.LogException(ex);
transaction.Rollback();
}
conn.Close();
}
}
return response;
}
now on response.Ref1 = Convert.ToString(p.Get<int>("#SP_ID")); line in some of my procedure I am getting SP_ID as output parameter and in some I am not getting SP_ID as output parameter
but the problem is when I am not returning SP_ID as output parameter I am getting error of
The given key was not present in the dictionary.
I want to check the key before execution of p.get<int>()
how can I do this?
So I fixed this by myself and thanks to #MarcGravell.
I declared a parameter in my DapperClass where I am using common ExecuteAsync method.
private DynamicParameters _Param;
public DapperClass()
{
_Param = new DynamicParameters();
}
now before transaction.Commit() line I am assigning the value to my parameter _Param = p;
and I created a public method with return type of DynamicParameters like below
public DynamicParameters GetDynamicParameters()
{
return _Param;
}
and also added a code like below from where I am executing my common dapper class
SPResponse response = await _Dapper.ExecuteAsync("[dbo].[TemplateAdd]", _DynamicParameter);
if (response.ReturnStatus == "0")
{
DynamicParameters dp = _Dapper.GetDynamicParameters();
response.Ref1 = Convert.ToString(dp.Get<int>("#SP_ID"));
response.Ref2 = request.FileServerId;
}
I am using springdoc-openapi with the latest version (1.3.0). Now I would like sort my tags in the UI by "name" property.
I know about the "springdoc.swagger-ui.tagsSorter" configuration and that I can use a custom sorter function. But I cannot find examples how the function should look like.
I tried the following which does not seem to work:
springdoc.swagger-ui.tagsSorter=(a, b) => a.get("name").localeCompare(b.get("name"))
By default, you can sort tags alphabetically:
https://springdoc.org/faq.html#how-can-i-sort-endpoints-alphabetically
You can have control on the tags order, using OpenApiCustomiser and define your own Comparator:
#Bean
public OpenApiCustomiser sortTagsAlphabetically() {
return openApi -> openApi.setTags(openApi.getTags()
.stream()
.sorted(Comparator.comparing(tag -> StringUtils.stripAccents(tag.getName())))
.collect(Collectors.toList()));
}
With reference from #brianbro's answer, as suggested at https://springdoc.org/faq.html#how-can-i-sort-endpoints-alphabetically
I added
#Tag(name="1. Admin endpoints")
#Tag(name = "2. Everyone's enpoints!")
and below prop to application.yml :
springdoc.swagger-ui.tagsSorter=alpha
And can see them sorted according to numbering on my swagger UI.
For sorting schemas , paths and tags in OpenApi.
#Bean
public OpenApiCustomiser openApiCustomiser() {
return openApi -> {
Map<String, Schema> schemas = openApi.getComponents().getSchemas();
openApi.getComponents().setSchemas(new TreeMap<>(schemas));
};
}
#Bean
public OpenApiCustomiser sortPathsAndTagsAlphabetically() {
return openApi -> {
Map<String, PathItem> paths = openApi.getPaths();
Paths sortedPaths = new Paths();
TreeMap<String, PathItem> sortedTree = new TreeMap<String, PathItem>(paths);
Set<Map.Entry<String, PathItem>> pathItems = sortedTree.entrySet();
Map<String, Map.Entry<String, PathItem>> distinctTagMap = new TreeMap<String, Map.Entry<String, PathItem>>();
for ( Map.Entry<String, PathItem> entry:pathItems) {
PathItem pathItem = entry.getValue();
Operation getOp = pathItem.getGet();
if(getOp != null) {
String tag = getOp.getTags().get(0);
if (!distinctTagMap.containsKey(tag)) {
distinctTagMap.put(tag, entry);
}
}
Operation postOp = pathItem.getPost();
if(postOp != null){
String tag1 = postOp.getTags().get(0);
if(!distinctTagMap.containsKey(tag1)){
distinctTagMap.put(tag1,entry);
}
}
Operation putOp = pathItem.getPut();
if(putOp != null) {
String tag2 = putOp.getTags().get(0);
if (!distinctTagMap.containsKey(tag2)) {
distinctTagMap.put(tag2, entry);
}
}
}
LinkedHashMap<String, PathItem> customOrderMap = new LinkedHashMap<String, PathItem>();
for (Map.Entry<String, PathItem> entry: distinctTagMap.values()) {
customOrderMap.put(entry.getKey(), entry.getValue());
}
for(Map.Entry<String, PathItem> entry : sortedTree.entrySet()) {
customOrderMap.putIfAbsent(entry.getKey(), entry.getValue());
}
sortedPaths.putAll(customOrderMap);
openApi.setPaths(sortedPaths);
};
}
Say my users subscribe to a plan. Is it possible then using Spring Cloud Gateway to rate limit user requests based up on the subscription plan? Given there're Silver and Gold plans, would it let Silver subscriptions to have replenishRate/burstCapacity of 5/10 and Gold 50/100?
I naively thought of passing a new instance of RedisRateLimiter (see below I construct a new one with 5/10 settings) to the filter but I needed to get the information about the user from the request somehow in order to be able to find out whether it is Silver and Gold plan.
#Bean
public RouteLocator myRoutes(RouteLocatorBuilder builder) {
return builder.routes()
.route(p -> p
.path("/get")
.filters(f ->
f.requestRateLimiter(r -> {
r.setRateLimiter(new RedisRateLimiter(5, 10))
})
.uri("http://httpbin.org:80"))
.build();
}
Am I trying to achieve something that is even possible with Spring Cloud Gateway? What other products would you recommend to check for the purpose if any?
Thanks!
Okay, it is possible by creating a custom rate limiter on top of RedisRateLimiter class. Unfortunately the class has not been architected for extendability so the solution is somewhat "hacky", I could only decorate the normal RedisRateLimiter and duplicate some of its code in there:
#Primary
#Component
public class ApiKeyRateLimiter implements RateLimiter {
private Log log = LogFactory.getLog(getClass());
// How many requests per second do you want a user to be allowed to do?
private static final int REPLENISH_RATE = 1;
// How much bursting do you want to allow?
private static final int BURST_CAPACITY = 1;
private final RedisRateLimiter rateLimiter;
private final RedisScript<List<Long>> script;
private final ReactiveRedisTemplate<String, String> redisTemplate;
#Autowired
public ApiKeyRateLimiter(
RedisRateLimiter rateLimiter,
#Qualifier(RedisRateLimiter.REDIS_SCRIPT_NAME) RedisScript<List<Long>> script,
ReactiveRedisTemplate<String, String> redisTemplate) {
this.rateLimiter = rateLimiter;
this.script = script;
this.redisTemplate = redisTemplate;
}
// These two methods are the core of the rate limiter
// Their purpose is to come up with a rate limits for given API KEY (or user ID)
// It is up to implementor to return limits based up on the api key passed
private int getBurstCapacity(String routeId, String apiKey) {
return BURST_CAPACITY;
}
private int getReplenishRate(String routeId, String apiKey) {
return REPLENISH_RATE;
}
public Mono<Response> isAllowed(String routeId, String apiKey) {
int replenishRate = getReplenishRate(routeId, apiKey);
int burstCapacity = getBurstCapacity(routeId, apiKey);
try {
List<String> keys = getKeys(apiKey);
// The arguments to the LUA script. time() returns unixtime in seconds.
List<String> scriptArgs = Arrays.asList(replenishRate + "", burstCapacity + "",
Instant.now().getEpochSecond() + "", "1");
Flux<List<Long>> flux = this.redisTemplate.execute(this.script, keys, scriptArgs);
return flux.onErrorResume(throwable -> Flux.just(Arrays.asList(1L, -1L)))
.reduce(new ArrayList<Long>(), (longs, l) -> {
longs.addAll(l);
return longs;
}) .map(results -> {
boolean allowed = results.get(0) == 1L;
Long tokensLeft = results.get(1);
Response response = new Response(allowed, getHeaders(tokensLeft, replenishRate, burstCapacity));
if (log.isDebugEnabled()) {
log.debug("response: " + response);
}
return response;
});
}
catch (Exception e) {
/*
* We don't want a hard dependency on Redis to allow traffic. Make sure to set
* an alert so you know if this is happening too much. Stripe's observed
* failure rate is 0.01%.
*/
log.error("Error determining if user allowed from redis", e);
}
return Mono.just(new Response(true, getHeaders(-1L, replenishRate, burstCapacity)));
}
private static List<String> getKeys(String id) {
String prefix = "request_rate_limiter.{" + id;
String tokenKey = prefix + "}.tokens";
String timestampKey = prefix + "}.timestamp";
return Arrays.asList(tokenKey, timestampKey);
}
private HashMap<String, String> getHeaders(Long tokensLeft, Long replenish, Long burst) {
HashMap<String, String> headers = new HashMap<>();
headers.put(RedisRateLimiter.REMAINING_HEADER, tokensLeft.toString());
headers.put(RedisRateLimiter.REPLENISH_RATE_HEADER, replenish.toString());
headers.put(RedisRateLimiter.BURST_CAPACITY_HEADER, burst.toString());
return headers;
}
#Override
public Map getConfig() {
return rateLimiter.getConfig();
}
#Override
public Class getConfigClass() {
return rateLimiter.getConfigClass();
}
#Override
public Object newConfig() {
return rateLimiter.newConfig();
}
}
So, the route would look like this:
#Component
public class Routes {
#Autowired
ApiKeyRateLimiter rateLimiter;
#Autowired
ApiKeyResolver apiKeyResolver;
#Bean
public RouteLocator theRoutes(RouteLocatorBuilder b) {
return b.routes()
.route(p -> p
.path("/unlimited")
.uri("http://httpbin.org:80/anything?route=unlimited")
)
.route(p -> p
.path("/limited")
.filters(f ->
f.requestRateLimiter(r -> {
r.setKeyResolver(apiKeyResolver);
r.setRateLimiter(rateLimiter);
} )
)
.uri("http://httpbin.org:80/anything?route=limited")
)
.build();
}
}
Hope it saves a work day for somebody...
This code has been simplified for this example.
The query is actually returned from a service, which is why I would prefer to write the method this way.
[HttpGet]
public PageResult<ExceptionLog> Logging(ODataQueryOptions<ExceptionLog> options)
{
var query = from o in _exceptionLoggingService.entities.ExceptionDatas
select new ExceptionLog {
ExceptionDataId = o.ExceptionDataId,
SiteId = o.SiteId,
ExceptionDateTime = o.ExceptionDateTime,
StatusCode = o.StatusCode,
Url = o.Url,
ExceptionType = o.ExceptionType,
ExceptionMessage = o.ExceptionMessage,
Exception = o.Exception,
RequestData = o.RequestData
};
var results = options.ApplyTo(query) as IEnumerable<ExceptionLog>;
var count = results.LongCount();
return new PageResult<ExceptionLog>(results, Request.GetNextPageLink(), count);
}
The above code errors on "results.LongCount()" with the following Exception:
SqlException: The text, ntext, and image data types cannot be compared or sorted, except when using IS NULL or LIKE operator.
It appears that I'm getting an exception with when trying to page, like this "$top=2". Everything works fine if my querystring is like this "$filter=ExceptionDataId gt 100".
Since ExceptionData (the Entity) matches ExceptionLog (business model) I can do something like this as a workaround:
[HttpGet]
public PageResult<ExceptionLog> Logging(ODataQueryOptions<ExceptionData> options)
{
var query = from o in _exceptionLoggingService.entities.ExceptionDatas
orderby o.ExceptionDateTime descending
select o;
var results = from o in options.ApplyTo(query) as IEnumerable<ExceptionData>
select new ExceptionLog {
ExceptionDataId = o.ExceptionDataId,
SiteId = o.SiteId,
ExceptionDateTime = o.ExceptionDateTime,
StatusCode = o.StatusCode,
Url = o.Url,
ExceptionType = o.ExceptionType,
ExceptionMessage = o.ExceptionMessage,
Exception = o.Exception,
RequestData = o.RequestData
};
return new PageResult<ExceptionLog>(results, Request.GetNextPageLink(), results.LongCount());
}
But this doesn't completely work for me because it's a little hackish and I can't use the service's method which already gives me an IQueryable.
Another thing to note, is if the Logging method is converted to IQueryable, everything works correctly. But I need to return the Count with the query so I have to return a PageResult.
This is the workaround I'm using. I only apply the filter from the ODataQueryOptions and I manually apply the Top and Skip.
First I created some extension methods:
using System;
using System.Collections.Generic;
using System.Linq;
namespace System.Web.Http.OData.Query
{
public static class ODataQuerySettingsExtensions
{
public static IEnumerable<T> ApplyFilter<T>(this IQueryable<T> query, ODataQueryOptions<T> options)
{
if (options.Filter == null)
{
return query;
}
return options.Filter.ApplyTo(query, new ODataQuerySettings()) as IEnumerable<T>;
}
public static IEnumerable<T> ApplyTopAndTake<T>(this IEnumerable<T> query, ODataQueryOptions<T> options)
{
IEnumerable<T> value = query;
if (options.Top != null)
{
value = value.Take(options.Top.Value);
}
if (options.Skip != null)
{
value = value.Skip(options.Skip.Value);
}
return value;
}
}
}
Now my method looks like this:
[HttpGet]
public PageResult<ExceptionLog> Logging(ODataQueryOptions<ExceptionLog> options)
{
// GetLogs returns an IQueryable<ExceptionLog> as seen in Question above.
var query = _exceptionLoggingService.GetLogs()
.ApplyFilter(options);
var count = query.Count();
var results = query.ApplyTopAndTake(options);
return new PageResult<ExceptionLog>(results, Request.GetNextPageLink(), count);
}
I have trained mahout model for three categories Category_A,Category_B,Category_C using 20newsGroupExample , Now i want to classify my documents using this model. Can somebody help me to understand output i am getting from this model.
Here is my output
{0:-2813549.8786637094,1:-2651723.736745838,2:-2710651.7525975127}
According to output category of document is 1, But expected category is 2. Am i going right or something is missing in my code ?
public class NaiveBayesClassifierExample {
public static void loadClassifier(String strModelPath, Vector v)
throws IOException {
Configuration conf = new Configuration();
NaiveBayesModel model = NaiveBayesModel.materialize(new Path(strModelPath), conf);
AbstractNaiveBayesClassifier classifier = new StandardNaiveBayesClassifier(model);
Vector st = classifier.classifyFull(v);
System.out.println(st.asFormatString());
System.out.println(st.maxValueIndex());
st.asFormatString();
}
public static Vector createVect() throws IOException {
FeatureVectorEncoder encoder = new StaticWordValueEncoder("text");
Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_36);
String inputData=readData();
StringReader in = new StringReader(inputData);
TokenStream ts = analyzer.tokenStream("body", in);
CharTermAttribute termAtt = ts.addAttribute(CharTermAttribute.class);
Vector v1 = new RandomAccessSparseVector(100000);
while (ts.incrementToken()) {
char[] termBuffer = termAtt.buffer();
int termLen = termAtt.length();
String w = new String(termBuffer, 0, termLen);
encoder.addToVector(w, 1.0, v1);
}
v1.normalize();
return v1;
}
private static String readData() {
// TODO Auto-generated method stub
BufferedReader reader=null;
String line, results = "";
try{
reader = new BufferedReader(new FileReader("c:\\inputFile.txt"));
while( ( line = reader.readLine() ) != null)
{
results += line;
}
reader.close();
}
catch(Exception ex)
{
ex.printStackTrace();
}
return results;
}
public static void main(String[] args) throws IOException {
Vector v = createVect();
String mp = "E:\\Final_Model\\model";
loadClassifier(mp, v);
}
}