I've been using ORMLite for close to a year now and I find it is excellent, it's simple and it works.
I have a project requirement to implement a broad search function- the user enters a value and ANY matching field in the table should be returned (POJOs). Tables have a simple 1:1 relationship with Java objects.
I can achieve this for a KNOWN class using the queryBuilder().where().like(...) however, I would like to build a generic dynamic function for all child classes.
All my persistent classes derive from a "Persistent" base class which has some basic housekeeping, handles store and delete for an individual object. This is where I am putting the "find" function, which ideally returns either an Iterator or a List<T>, preferably an Iterator.
I've already got a Map<String, DataType> for all DatabaseField annotated Fields. Right now, I just want a "brute" force is string "X" present anywhere in field Y. I'll figure out data type conversions and checking later. If I can build the query dynamically, then I can resolve the data typing issue in the next version.
I've looked at the methods that take maps as parameters and they are a narrow match, i.e. all must match. I am looking for the opposite: any match will do.
The only method I can think of is to build individual "like" components and string them together into a larger statement, from the documentation, I can't figure out how to do that dynamically.
Any suggestions welcome.
OK, I've messed with some code, and this achieves what I want- but I would very much welcome comments to improve or suggest better ways of achieving the intended outcome.
protected QueryBuilder<T, Long> getQuery(String text, boolean wide, boolean count) {
if (null == fields)
reflectKeyInfo();
QueryBuilder<T, Long> qb = null;
if (count)
qb = getDao().queryBuilder().setCountOf(true);
else
qb = getDao().queryBuilder();
if (null == qb)
return null;
Where<T, Long> where = qb.where();
boolean equals = text.contains("=") ;
boolean like = text.contains("~") ;
if (equals || like) {
String field = null ;
String value = null ;
try {
if (equals) {
field = Strings.left(text, text.indexOf("=")) ;
value = Strings.right(text, text.indexOf("=")) ;
if (Strings.isValid(field) && Strings.isValid(value))
where.eq(field, value) ;
}
else
if (like) {
field = Strings.left(text, text.indexOf("~")) ;
value = Strings.right(text, text.indexOf("~")) ;
if (Strings.isValid(field) && Strings.isValid(value))
where.like(field, "%" + value + "%") ;
}
}
catch (SQLException e) {
lastError = GlobalConfig.log(e, true) ;
Dialogs.alert("Query Failed",
"<br />Unable to extract field and value from " +
field + "=" + value + "<br />" + lastError) ;
}
}
else
for (String field : fields.keySet()) {
try {
if (fields.get(field).isAssignableFrom(String.class)) {
if (wide)
where.like(field, "%" + text + "%");
else
where.eq(field, text);
where.or();
}
}
catch (SQLException e) {
lastError = GlobalConfig.log(e, true);
}
} // for
return qb;
}
Related
Im using Devexpress Grid to show my result for final users.
since im using remote mode , i should filter my own result on server based on grid fiter.
I made custom JS swtich\case to make SQL where condition based on filter result like this :
let whereClause = '';
if (loadOptions['filter']) {
const rawFilter = loadOptions['filter'];
let filters = null;
if (_.isArray(rawFilter[0])) {
filters = _.map(rawFilter, (item) => {
if (_.isArray(item)) {
if (!_.isNumber(item[2])) {
switch (item[1]) {
case 'contains':
return `(${item[0]} LIKE N'%${item[2]}%')`;
case 'notcontains':
return `(${item[0]} NOT LIKE N'%${item[2]}%')`;
case 'startswith':
return `(${item[0]} LIKE N'${item[2]}%')`;
case 'endswith':
return `(${item[0]} LIKE N'%${item[2]}')`;
default:
return `(${item[0]} ${item[1]} N'${item[2]}')`;
}
}
else {
return `(${item[0]} ${item[1]} ${item[2]})`;
}
}
else {
return item;
}
});
whereClause = _.join(filters, ' ');
}
}
And then passing the whereClause to controller , like : "( ColomnA like N'ABC')"
After that , using it in Store Procedure like this (passing it as SearchParam):
CREATE OR ALTER PROCEDURE GetTestResult
#Skip INT = 0,
#Take INT = 400000,
#SearchParam NVARCHAR(MAX) = null
AS
BEGIN
SET NOCOUNT ON;
DECLARE #sqlCommand NVARCHAR(MAX)
DECLARE #sqlCommandPagination NVARCHAR(MAX)
DECLARE #sqlCommandFinall NVARCHAR(MAX)
SET #sqlCommand = 'SELECT ColomnA,ColomnB,ColomnC FROM dbo.TestTable '
SET #sqlCommandPagination = ' ORDER BY ColomnA asc OFFSET ' + CAST(#Skip as varchar(500)) +
' ROWS FETCH NEXT ' + CAST(#Take as varchar(500)) + ' ROWS ONLY;'
If( #SearchParam <> '' AND #SearchParam IS NOT NULL )
Set #sqlCommandFinall = #sqlCommand + ' Where ' + #SearchParam + #sqlCommandPagination;
If( #SearchParam = '' OR #SearchParam IS NULL )
Set #sqlCommandFinall = #sqlCommand + #sqlCommandPagination;
Execute SP_ExecuteSQL #sqlCommandFinall;
END;
GO
Is there any way to make this method Sql Injection proof ?
Im Using Dapper\SqlMapper\Query Function For calling my sp from C#.
Not the way it's designed. Or at least not without carefully parsing the statement and then leveraging parameterized values and/or positively validating the components. And doing that is much more complex and prone to error than just passing the components.
In your current design, it's trivial to inject SQL into the where or order by. You should break this down into its components, pass those through and validate them at the backend. So, rather than passing ColomnA like N'ABC' (sic) you should pass ColomnA (sic) and ABC seperately, and use parameterized queries for the values (bind them).
In the second case, many databases will not allow parameterizing order by values, so you should be sure to use positive validation in that case.
I am trying to make a Dynamic search for EF Core. I made the whole thing in a loop like this:
foreach (var i in vm.SearchProperties)
{
if (result == null)
result = db.MyTable.Where(x => x.GetType().GetProperty(i).GetValue(x, null).ToString().ToLower().StartsWith("MySearchString"));
else
result = result.Where(x => x.GetType().GetProperty(i).GetValue(x,null).ToString().ToLower().StartsWith(i.Suchfeld.ToLower(“mySearchString”)));
}
Before I added the reflection part it runs pretty fast. As soon I added the Reflection to it, it got slowed down by a factor of 1000. Any ideas how I get it speeded up or around the reflection part.
Your expression is too complex to be interpreted by the LINQ to SQL converter, so it is being compiled and executed on every item in your table, so it's no surprise it executes exceedingly slow.
You need to build an expression tree based on which properties you want to search, then construct an Expression<Func<MyType, bool>> to pass to your Where(...) method. That way the LINQ to SQL converter recognizes it.
Try this:
ParameterExpression param = Expression.Parameter(typeof(MyType));
MethodInfo stringStartsWith = typeof(string).GetMethods().First(m => m.Name == "StartsWith" && m.GetParameters().Length == 1);
PropertyInfo firstProp = typeof(MyType).GetProperty(vm.SearchProperties.First());
MemberExpression firstMembAccess = Expression.Property(param, firstProp);
MethodCallExpression firstStartsWithExpr = Expression.Call(firstMembAccess, stringStartsWith, Expression.Constant(mySearchString));
Expression current = firstStartsWithExpr;
foreach (string s in vm.SearchProperties.Skip(1))
{
PropertyInfo prop = typeof(MyType).GetProperty(s);
MemberExpression membAccess = Expression.Property(param, prop);
MethodCallExpression startsWithExpr = Expression.Call(membAccess, stringStartsWith, Expression.Constant(mySearchString));
current = Expression.OrElse(current, startsWithExpr);
}
Expression<Func<MyType, bool>> mySearchExpression = Expression.Lambda<Func<MyType, bool>>(current, param);
result = db.MyTable.Where(mySearchExpression);
Note: MyType refers to whatever your entity type is.
As people already answered, reflection cannot be used in IQueryable.
Data are loaded in the memory and the Where clause is performed.
It's similar as if you have done the following code (adding ToList() before the where clause):
foreach (var i in vm.SearchProperties)
{
if (result == null)
result = db.MyTable.ToList().Where(x => x.GetType().GetProperty(i).GetValue(x, null).ToString().ToLower().StartsWith("MySearchString"));
else
result = result.ToList().Where(x => x.GetType().GetProperty(i).GetValue(x,null).ToString().ToLower().StartsWith(i.Suchfeld.ToLower(“mySearchString”)));
}
#Mr Anderson is right. You need to use expression tree instead of reflection.
However, using expression tree may be sometimes to much complex to build.
Disclaimer: I'm the owner of the project Eval-Expression.NET
This project allows you to evaluate and execute dynamic expression at runtime.
In short, you can pass a dynamic string to the where clause.
Wiki: Eval-Expression.NET - LINQ Dynamic
foreach (var i in vm.SearchProperties)
{
if (result == null)
result = db.MyTable.Where(x => "x." + i + ".ToLower().StartsWith('MySearchString')");
else
result = result.Where(x => "x." + i + ".ToLower().StartsWith('MySearchString')");
}
I am trying to query my database to get a specific data from my database. however when I convert the query to string it doesn't return the select value, instead it returns the whole SQL Query in a string. I am stumped on why this is happening
public ActionResult StudiedModules()
{
Model1 studiedModules = new Model1();
List<StudiedModulesModel> listModules = new List<StudiedModulesModel>();
using (EntityOne context = new EnitityOne())
{
foreach(var module in context.StudiedModules){
studiedModules.School = context.ModuleDatas.Where(p=>p.ModuleCode == module.ModuleCode).Select(u=>u.School).ToString();
studiedModules.Subject = context.ModuleDatas.Where(p=>p.ModuleCode == module.ModuleCode).Select(u=>u.Subject).ToString();
}
}
var data = listModules;
return View(data);
}
Calling ToString() on an Entity Framework Linq query like that will in fact return the SQL Query. So as it's written, your code is doing exactly what you wrote it to do.
If you want to select the first result from the IQueryable<T>, then you need to call First() before your ToString(). So, try changing
studiedModules.School = context.ModuleDatas.Where(p=>p.ModuleCode == module.ModuleCode).Select(u=>u.School).ToString();
studiedModules.Subject = context.ModuleDatas.Where(p=>p.ModuleCode == module.ModuleCode).Select(u=>u.Subject).ToString()
to
studiedModules.School = context.ModuleDatas.Where(p=>p.ModuleCode == module.ModuleCode).Select(u=>u.School).First().ToString();
studiedModules.Subject = context.ModuleDatas.Where(p=>p.ModuleCode == module.ModuleCode).Select(u=>u.Subject).First().ToString()
There are a whole lot more methods available depending on what you're trying to accomplish. If you want to get a list, use ToList(), or as Uroš Goljat pointed out, you can get a comma-separated list of values using the Aggregate( (a, b)=> a + ", " + b) method.
How about using Aggregate( (a, b)=> a + ", " + b) instead of ToString().
Regards,
Uros
I found the solution Burt Beckwith offered in this question: add user define properties to a domain class and think this could be a viable option for us in some situations. In testing this, I have a domain with a Map property as described in the referenced question. Here is a simplified version (I have more non-map properties, but they are not relevant to this question):
class Space {
String spaceDescription
String spaceType
Map dynForm
String toString() {
return (!spaceType)?id:spaceType + " (" + spaceDescription + ")"
}
}
I have some instances of space saved with some arbitrary data in the dynForm Map like 'Test1':'abc' and 'Test2':'xyz'.
I am trying to query this data and have succesfully used HQL to filter doing the following:
String className = "Space"
Class clazz = grailsApplication.domainClasses.find { it.clazz.simpleName == className }.clazz
def res = clazz.executeQuery("select distinct space.id, space.spaceDescription from Space as space where space.dynForm['Test1'] = 'abc' " )
log.debug "" + res
I want to know if there is a way to select an individual item from the Map dynForm in a select statement. Somehting like this:
def res = clazz.executeQuery("select distinct space.id, elements(space.dynForm['Test1']) from Space as space where space.dynForm['Test1'] = 'abc' " )
I can select the entire map like this:
def res = clazz.executeQuery("select distinct elements(space.dynForm) from Space as space where space.dynForm['Test1'] = 'abc' " )
But I just want to get a specific instance based on the string idx.
how about to use Criteria, but i don't have any sql server so i haven't tested yet.
def results = Space.createCriteria().list {
dynForm{
like('Test1', 'abc')
}
}
I have a listview that I fill from an Adapter. My original code the data was being returned from a table, but now I need to get the code from a query with a join so the examples I used will no longer work and I haven't been able to find out how to use a query for this. I'm using an ORMrepository.
In my ORMrepository I have this function
public IList<Coe> GetmyCoe()
{
using (var database = new SQLiteConnection(_helper.WritableDatabase.Path))
{
string query = "SELECT Coe.Id, Adult.LName + ', ' + Adult.MName AS Name, Coe.Createdt FROM Adult INNER JOIN Coe ON Adult.CoeMID = Coe.Id";
return database.Query<Coe>(query);
}
}
which actually returns the data I want.
then in my Activity page I have this.
_list = FindViewById<ListView>(Resource.Id.List);
FindViewById<ListView>(Resource.Id.List).ItemClick += new System.EventHandler<ItemEventArgs>(CoeList_ItemClick);
var Coe = ((OmsisMobileApplication)Application).OmsisRepository.GetmyCoe();
_list.Adapter = new CoeListAdapter(this, Coe);
My Adapter page is where I have the problem, I know it is set up to to looking at a table which I'm not doing anymore. But I don't know how to change it for what I'm passing into it now. Current CoeListAdapter is:
public class CoeListAdapter : BaseAdapter
{
private IEnumerable<Coe> _Coe;
private Activity _context;
public CoeListAdapter(Activity context, IEnumerable<Coe> Coe)
{
_context = context;
_Coe = Coe;
}
public override View GetView(int position, View convertView, ViewGroup parent)
{
var view = (convertView
?? _context.LayoutInflater.Inflate(
Resource.Layout.CoeListItem, parent, false)
) as LinearLayout;
var Coe = _Coe.ElementAt(position);
view.FindViewById<TextView>(Resource.Id.CoeMID).Text = Coe.Id.ToString();
//view.FindViewById<TextView>(Resource.Id.GrdnMaleName).Text = Coe.Name;
view.FindViewById<TextView>(Resource.Id.CreateDt).Text = Coe.CreateDt;
return view;
}
public override int Count
{
get { return _Coe.Count(); }
}
public Coe GetCoe(int position)
{
return _Coe.ElementAt(position);
}
public override Java.Lang.Object GetItem(int position)
{
return null;
}
public override long GetItemId(int position)
{
return position;
}
}
How do I set up the CoeListAdapter.cs page so that it can use the passed in data. As you can see I have a commented out lines where I fill a TextView which error because Coe.Name is not in the table model for Coe. but it is returned in the query. I believe my problem is IEnumerable but what do I change it to. I'm new to Mobile developement and suing VS2010 for Mono
The problem probably lies with the binding/mapping of the object not the creation of the view.
Or probably more specifically, the query.
Adult.LName + ', ' + Adult.MName AS Name
this should be:
Adult.LName || ', ' || Adult.MName AS Name
See also: String concatenation does not work in SQLite
From the sqlite docs: http://www.sqlite.org/lang_expr.html under the Operators heading:
The unary operator + is a no-op. It can be applied to strings,
numbers, blobs or NULL and it always returns a result with the same
value as the operand.
Note that there are two variations of the equals and not equals
operators. Equals can be either = or ==. The non-equals operator can
be either != or <>. The || operator is "concatenate" - it joins
together the two strings of its operands. The operator % outputs the
value of its left operand modulo its right operand.
The result of any binary operator is either a numeric value or NULL,
except for the || concatenation operator which always evaluates to
either NULL or a text value.
This shows that the + will evaluate to zero. If you use ||, the value will either be the correct value or NULL (if either of Adult.LName or Adult.MName is NULL).
This can be fixed by:
coalesce(first, '') || ', ' || coalesce(second, '')
but this may result in , LastName or FirstName,.
Another way would be to create another two properties in Coe called LName and MName.
Then bind the values to those properties and use the Name property like this:
public string Name
{
get { return string.Join(", ", LName, MName); }
}
This will probably be better as you can change how the Name appears especially if there are different combinations of First, Middle and Last names in different places.
And off topic:
I believe my problem is IEnumerable...
This is probably not too true as it returns the correct values. A better way would be to use IList as IEnumerable will iterate through the list each time to get the item as it does not know that the collection is actually a list. (I think)
thanks for the help on the concantination, I did find that was wrong, I did fix my problem, I was using an example by Greg Shackles on how to set up using a data base. what I had to do was create a new model with the elements I was wanting. So I created a new Model and called it CoeList, then everywhere I had List or IEnumerable I changed it to List or IEnumerable and it worked.