I have a subresource like 'X/:parent_id/Y?=where{'a':'b'}'. This it gets transformed to a MongoDB query as $and:[{'parent_id},{'a':'b'}].
Can I convert it to $or:[{'parent_id},{'a':'b'}]?
I have tried with a pre_GET operation and altering the lookup but the :parent_id value it is not populated yet in the lookup (if it ever does), and the database fetch hook is for after the data has been fetched, so is not a solution.
Really appreciated.
Related
So, I have multiple tables in multiple databases.
The user sends me database details, table names, columns to be selected and which column to use as condition for join. The structure is similar to:
[{ database, table_name, join_column_name }]
My current implementation is like this,
Connect to the first DB
Fetch all data from the table
Store the data in a result variable (PG::Result instance)
Extract all unique values in current join_column_name from result to a filter_values array
Connect to next database
Fetch all data where value of current join_column_name is in filter_values
Store the data to a local_result variable (PG::Result instance)
Simulate inner join on result and local_result objects, and store in result
Repeat from step 4
Also, The join_column_name may or may not be an indexed column
In step 8, I have to create a new PG::Result object, and store mapped data from result and local_result objects. How do I attain this?
Is there a better way to do this, without all these custom logic?
I've looked into disable_joins: true, but don't know if it is applicable in this case.
When I update an object in my sqlite API with ajax, it keeps the order of my object array - so the front end looks the same. When I update an object in the API after switching the db to postgres, it changes the order of the array - mostly placing the updated objects at the end of the array. Any ideas what's going on here?
I've tried deleting and remaking the database, no luck. I switched back to sqlite and is working normally again.
In SQL order is not guaranteed. If you desire a particular order, the safest thing to do is to add a sort key to your records, and make sure you're doing an ORDER BY on your select statement.
The fact that SQLite is preserving your ordering is kind of a "mistake" of implementation. You should not rely on the engine to do anything outside the specification.
Quote from the Postgres docs:
After a query has produced an output table (after the select list has been processed) it can optionally be sorted. If sorting is not chosen, the rows will be returned in an unspecified order. The actual order in that case will depend on the scan and join plan types and the order on disk, but it must not be relied on. A particular output ordering can only be guaranteed if the sort step is explicitly chosen.
That said: without an explicit ORDER clause the order of the returned records is kind of random.
I'd like to know the best way to retrieve the schema for a query's result WITHOUT executing the query. By Schema I want to know the name of the fields that will be returned and if they are boolean, literal or float, ...
For those familiar with SQL Server I want the same kind of behavior than using a FMTONLY.
At the moment my best approach is to perform a LIMIT 0 ... but anyway I'm nowhere to know the type of the returned fields.
What you are seeking is not generally possible.
Unlike SQL, neo4j is "schemaless", and the "schema" of a query result is generally dependent on the data in the DB. This means that, generally, a query's result schema cannot be known without actually executing the query.
Quick question (hopefully)
I have a large dataset (>100,000 records) that I would like to use as a lookup to determine existence or non-existence of multiple keys. The purpose of this is to find FK violations before trying to commit them to the database to try and avoid the resultant EDatabaseError messing up my transaction.
I had been using TClientDataSet/TDatasetProvider with the FindKey method, as this allowed a client-side index to be set up and was faster (2s to scan each key rather than 10s for ADO). However, moving to large datasets the population of the CDS is starting to take far more time than the local index is saving.
I see that I have a few options for alternatives:
client cursor with TADOQuery.locate method
ADO SELECT statements for each check (no client cache)
ADO SEEK method
Extend TADOQuery to mimic FindKey
The Locate method seems easiest and doesn't spam the server with the SELECT/SEEK methods. I like the idea of extending the TADOQuery, but was wondering whether anyone knew of any ready-made solutions for this rather than having to create my own?
I would create a temporary table in the database server. Insert all 100,000 records into this temp table. Do bulk inserts of say 3000 records at a time, to minimise round trips to the server. Then run select statements on this temp table to check for foreign key violations etc. If all okay, do an insert SQL from the temp table to the main table.
I have ten master tables and one Transaction table. In my transaction table (it is a memory table just like ClientDataSet) there are ten lookup fields pointing to my ten master tables.
Now i am trying to dynamically assigning key field values to all my lookup key field values (of the transaction table) from a different Server(data is coming as a soap xml). Before assigning these values i need to check whether the corresponding result value is valid in master tables or not. I am using a filter (eg status = 1 ) to check whether it is valid or not.
Currently how we are doing is, before assigning each key field value we are filtering the master tables using this filter and using the locate function to check whether it is there or not. and if located we will assign its key field value.
This will work fine if there is only few records in my master tables. Consider my master tables having fifty thousand records each (yeah, customer is having so much data), this will lead to big performance issue.
Could you please help me to handle this situation.
Thanks
Basil
The only way to know if it is slow, why, where, and what solution works best is to profile.
Don't make a priori assumptions.
That being said, minimizing round trips to the server and the amount of data transferred is often a good thing to try.
For instance, if your master tables are on the server (not 100% clear from your question), sending only 1 Query (or stored proc call) passing all the values to check at once as parameters and doing a bunch of "IF EXISTS..." and returning all the answers at once (either output params or a 1 record dataset) would be a good start.
And 50,000 records is not much, so, as I said initially, you may not even have a performance problem. Check it first!