I have a list of data with a title column (among many other columns) and I have a Power BI parameter that has, for example, a value of "a,b,c". What I want to do is loop through the parameter's values and remove any rows that begin with those characters.
For example:
Title
a
b
c
d
Should become
Title
d
This comma separated list could have one value or it could have twenty. I know that I can turn the parameter into a list by using
parameterList = Text.Split(<parameter-name>,",")
but then I am unsure how to continue to use that to filter on. For one value I would just use
#"Filtered Rows" = Table.SelectRows(#"Table", each Text.StartsWith([key], <value-to-filter-on>))
but that only allows one value.
EDIT: I may have worded my original question poorly. The comma separated values in the parameterList can be any number of characters (e.g.: a,abcd,foo,bar) and I want to see if the value in [key] starts with that string of characters.
Try using List.Contains to check whether the starting character is in the parameter list.
each List.Contains(parameterList, Text.Start([key], 1)
Edit: Since you've changed the requirement, try this:
Table.SelectRows(
#"Table",
(C) => not List.AnyTrue(
List.Transform(
parameterList,
each Text.StartsWith(C[key], _)
)
)
)
For each row, this transforms the parameterList into a list of true/false values by checking if the current key starts with each text string in the list. If any are true, then List.AnyTrue returns true and we choose not to select that row.
Since you want to filter out all the values from the parameter, you can use something like:
= Table.SelectRows(#"Changed Type", each List.Contains(Parameter1,Text.Start([Title],1))=false)
Another way to do this would be to create a custom column in the table, which has the first character of title:
= Table.AddColumn(#"Changed Type", "FirstChar", each Text.Start([Title],1))
and then use this field in the filter step:
= Table.SelectRows(#"Added Custom", each List.Contains(Parameter1,[FirstChar])=false)
I tested this with a small sample set and it seems to be running fine. You can test both and see if it helps with the performance. If you are still facing performance issues, it would probably be easier if you can share the pbix file.
This seems to work fairly well:
= List.Select(Source[Title], each Text.Contains(Parameter1,Text.Start(_,1))=false)
Replace Source with the name of your table and Parameter1 with the name of your Parameter.
I am struggling on this for a week now. I need to write an elasticsearch query api in java with following combinations:
Price range (0 to n number of ranges)
Age range (0 to n number of ranges)
Search text
Term Filters (can match more than one field
I want to have OR relationship within each group (e.g. price 0-10 or 11-15) but AND relationship among different groups (e.g. price 0-10 AND productType="clothing")
I have written the below code in Java, I have verified that the data exists for the range and filter I am testing with but I am getting zero resultset. I am not sure what is missing here.
#Override
public List<Shoppingi> search(SearchCriteria criteria, Pageable pageable) {
final NativeSearchQueryBuilder searchQuery = new NativeSearchQueryBuilder();
BoolQueryBuilder boolQuery = boolQuery();
// apply the text search
if (StringUtils.isNotBlank(criteria.getSearchText()))
boolQuery.must(multiMatchQuery(criteria.getSearchText(), FIELD_NAMES_FOR_TERM_SEARCH));
// apply the term filters
BoolQueryBuilder termsQuery = boolQuery();
criteria.getFilters().entrySet().stream().forEach(entry -> termsQuery.should(termsQuery(entry.getKey(), entry.getValue())));
// apply the range filters
BoolQueryBuilder rangeQuery = boolQuery();
criteria.getRangeFilters().forEach((key, value) -> rangeQuery.should(rangeQuery(key).gte(value.getLowerBound()).lt(value.getUpperBound())));
boolQuery.must(termsQuery);
boolQuery.must(rangeQuery);
searchQuery.withQuery(boolQuery);
searchQuery.withPageable(pageable);
NativeSearchQuery query = searchQuery.build();
log.info("query: {}", query.getQuery().toString());
return elasticsearchTemplate.queryForList(query,Shopping.class);
}
I annotated the price object as index.no, which caused the query to not work. Once I re-indexed my data with price as non_analyzed, my code started working. So it was nothing wrong with the code above, it was the way I configured the variable I was querying against.
I am trying to create a report in SSRS. Below is a small example of what my dataset looks like.
Example Data Set
So, there are three different stores (A,B,C) and each has a landlord (a,b,c). Landlords can pay via three different methods (1,2,3) and the amounts paid per method are shown.
Right now, I have two filters set up. The first is by Store and the second is by Landlord.
What I am having trouble with is:
How can I set up a filter by the Amount that will return information from an entire Store/Landlord?
So for example, if I wanted to filter Amount by 150, I would like to return all the "payment" information for the store(s) that have a payment of 150. Such as the following:
Desired Result
Is it possible to add a filter to return information from the entire group? (Store and Landlord are the group in this case)
I am new to SSRS so any help/insight would be greatly appreciated!
You can use LookUpSet to locate the matching groups, JOIN to put the results in a string and the INSTR function to filter your results.
=IIF(ISNOTHING(Parameters!AMOUNT.Value) OR INSTR(
Join(LOOKUPSET(Fields!Amount.Value, Fields!Amount.Value, Fields!Store.Value, "DataSet1"), ", ") ,
Fields!Store.Value
) > 0, 1, 0)
This translates to:
If the Store value is found (INSTR > 0) in the list (JOIN) of Stores where the Amount is the current Amount (Lookupset).
In your filter, put the above expression in the Expression, change the type to INTEGER and the Value to 1.
[
I want to replace the value of the 'Amount' key in a map (literal) with the sum of the existing 'Amount' value plus the new 'Amount' value such where both the 'type' and 'Price' match. The structure I have so far is:
WITH [{type:1, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},{Price:12,Amount:300}]},
{type:2, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},{Price:12,Amount:300}]},
{type:3, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},{Price:12,Amount:300}]}] as ExistingOrders,
{type:2, Order:{Price:11,Amount:50}} as NewOrder
(I'm trying to get it to:)
RETURN [{type:1, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},{Price:12,Amount:300}]},
{type:2, Orders:[{Price:10,Amount:100},{Price:11,Amount:250},{Price:12,Amount:300}]},
{type:3, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},{Price:12,Amount:300}]}] as CombinedOrders
If there is no existing NewOrder.type and NewOrder.Price then it should obviously insert the new record rather than add it together.
Sorry, this is possibly really straight forward, but I'm not very good at this yet.
thanks
Edit:
I should add, that I have been able to get this working for a simpler map structure as such:
WITH [{type:1, Amount:100},{type:2, Amount:200},{type:3, Amount:300}] as ExistingOrders,
{type:2, Amount:50} as NewValue
RETURN reduce(map=filter(p in ExistingOrders where not p.type=NewValue.type),x in [(filter(p2 in ExistingOrders where p2.type=NewValue.type)[0])]|CASE x WHEN null THEN NewValue ELSE {type:x.type,Amount:x.Amount+NewValue.Amount} END+map) as CombinedOrders
But I'm struggling I think because of the Orders[array] in my first example.
I believe you are just trying to update the value of the appropriate Amount in ExistingOrders.
The following query is legal Cypher, and should normally work:
WITH ExistingOrders, NewOrder, [x IN ExistingOrders WHERE x.type = NewOrder.type | x.Orders] AS eo
FOREACH (y IN eo |
SET y.Amount = y.Amount + CASE WHEN y.Price = NewOrder.Order.Price THEN NewOrder.Order.Amount ELSE 0 END
)
However, the above query produces a (somewhat) funny ThisShouldNotHappenError error with the message:
Developer: Stefan claims that: This should be a node or a relationship
What the message is trying to say (in obtuse fashion) is that you are not using the neo4j DB in the right way. Your properties are way too complicated, and should be separated out into nodes and relationships.
So, I will a proposed data model that does just that. Here is how you can create nodes and relationships that represent the same data as ExistingOrders:
CREATE (t1:Type {id:1}), (t2:Type {id:2}), (t3:Type {id:3}),
(t1)-[:HAS_ORDER]->(:Order {Price:10,Amount:100}),
(t1)-[:HAS_ORDER]->(:Order {Price:11,Amount:200}),
(t1)-[:HAS_ORDER]->(:Order {Price:12,Amount:300}),
(t2)-[:HAS_ORDER]->(:Order {Price:10,Amount:100}),
(t2)-[:HAS_ORDER]->(:Order {Price:11,Amount:200}),
(t2)-[:HAS_ORDER]->(:Order {Price:12,Amount:300}),
(t3)-[:HAS_ORDER]->(:Order {Price:10,Amount:100}),
(t3)-[:HAS_ORDER]->(:Order {Price:11,Amount:200}),
(t3)-[:HAS_ORDER]->(:Order {Price:12,Amount:300});
And here is a query that will update the correct Amount:
WITH {type:2, Order:{Price:11,Amount:50}} as NewOrder
MATCH (t:Type)-[:HAS_ORDER]->(o:Order)
WHERE t.id = NewOrder.type AND o.Price = NewOrder.Order.Price
SET o.Amount = o.Amount + NewOrder.Order.Amount
RETURN t.id, o.Price, o.Amount;
There's two parts to your question - one with a simple answer, and a second part that doesn't make sense. Let me take the simple one first!
As far as I can tell, it seems you're asking how to concatenate a new map on to a collection of maps. So, how to add a new item in an array. Just use + like this simple example:
return [{item:1}, {item:2}] + [{item:3}];
Note that the single item we're adding at the end isn't a map, but a collection with only one item.
So for your query:
RETURN [
{type:1, Orders:[{Price:10,Amount:100},
{Price:11,Amount:200},
{Price:12,Amount:300}]},
{type:2, Orders:[{Price:10,Amount:100},
{Price:11,Amount:**250**},
{Price:12,Amount:300}]}]
+
[{type:3, Orders:[{Price:10,Amount:100},
{Price:11,Amount:200},{Price:12,Amount:300}]}]
as **CombinedOrders**
Should do the trick.
Or you could maybe do it a bit cleaner, like this:
WITH [{type:1, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},{Price:12,Amount:300}]},
{type:2, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},{Price:12,Amount:300}]},
{type:3, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},{Price:12,Amount:300}]}] as ExistingOrders,
{type:2, Order:{Price:11,Amount:50}} as NewOrder
RETURN ExistingOrders + [NewOrder];
OK now for the part that doesn't make sense. In your example, it looks like you want to modify the map inside of the collection. But you have two {type:2} maps in there, and you're looking to merge them into something with one resulting {type:3} map in the output that you're asking for. If you need to deconflict map entries and change what the map entry ought to be, it might be that cypher isn't your best choice for that kind of query.
I figured it out:
WITH [{type:1, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},Price:12,Amount:300}]},{type:2, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},{Price:12,Amount:300}]},{type:3, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},{Price:12,Amount:300}]}] as ExistingOrders,{type:2, Orders:[{Price:11,Amount:50}]} as NewOrder
RETURN
reduce(map=filter(p in ExistingOrders where not p.type=NewOrder.type),
x in [(filter(p2 in ExistingOrders where p2.type=NewOrder.type)[0])]|
CASE x
WHEN null THEN NewOrder
ELSE {type:x.type, Orders:[
reduce(map2=filter(p3 in x.Orders where not (p3.Price=(NewOrder.Orders[0]).Price)),
x2 in [filter(p4 in x.Orders where p4.Price=(NewOrder.Orders[0]).Price)[0]]|
CASE x2
WHEN null THEN NewOrder.Orders[0]
ELSE {Price:x2.Price, Amount:x2.Amount+(NewOrder.Orders[0]).Amount}
END+map2 )]} END+map) as CombinedOrders
...using nested Reduce functions.
So, to start with it combines a list of orders without matching type, with a list of those orders (actually, just one) with a matching type. For those latter ExistingOrders (with type that matches the NewOrder) it does a similar thing with Price in the nested reduce function and combines non-matching Prices with matching Prices, adding the Amount in the latter case.
I have the following sample code. The objective is to run SQL statement with multiple input parameters.
[<Literal>]
let connectionString = #"Data Source=Localhost;Initial Catalog=Instrument;Integrated Security=True"
[<Literal>]
let query = "SELECT MacroName, MacroCode FROM Instrument WHERE MacroCode IN (#codeName)"
type MacroQuery = SqlCommandProvider<query, connectionString>
let cmd = new MacroQuery()
let res = cmd.AsyncExecute(codeName= [|"CPI";"GDP"|]) |> Async.RunSynchronously
However, codeName is inferred to be string type instead of an array or list and give me an error.
Alternatively, I could run the query without where statement and filter based on the result. However, in lots of other cases that returns millions of rows, I would prefer filter data at the SQL server level to be more efficient.
I didn't find any relevant samples on the documentation of fsharp.data.sqlclient. Please help!
"See Table-valued parameters (TVPs)" section in the documentation:
http://fsprojects.github.io/FSharp.Data.SqlClient/configuration%20and%20input.html
If you have an upper bound n on the values in the IN list, you could just make n parameters. If that's unmanageable, I'm afraid the TVP suggestion is your best option. The reason the FSharp.Data.SqlClient library is unlikely to ever support this directly is because the types are generated based on results from sp_describe_undeclared_parameters; there's no T-SQL parser. We had a single digit upper bound in this scenario and were loathe to change the database, so this option worked for us.
You can use STRING_SPLIT to abstract away the use of table-valued parameters. It seems you have to also declare the param first.
DECLARE #param varchar(1000) = #commaSeparatedList
SELECT Col1 FROM MyTable
WHERE Col2 IN (
SELECT value FROM STRING_SPLIT(#param, ',')
)